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Preface 


For the thirty-fourth year, the Research and Theory Division of the Association for Educational 
Communications and Technology (AECT) is sponsoring the publication of these Proceedings. Papers 
published in this volume were presented at the annual AECT Convention in Anaheim, CA. A limited 
quantity of these Proceedings were printed and sold in both hardcopy and electronic versions. Volumes 1 
and 2 are available through the Educational Resources Clearinghouse (ERIC) System. Proceedings volumes 
are available to members at AECT.ORG. 

The Proceedings of AECT’s Convention are published in two volumes. Volume #1 contains papers dealing 
primarily with research and development topics. Papers dealing with the practice of instructional 
technology including instruction and training issues are contained in Volume #2. This year, both volumes 
are included in one document. 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in these 
Proceedings were subjected to a reviewing process. All references to authorship were removed from 
proposals before they were submitted to referees for review. Approximately sixty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
publication in these Proceedings. The papers contained in this document represent some of the most current 
thinking in educational communications and technology. 


Michael R. Simonson 
Editor 
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Descriptors : game education 

The ways people entertain themselves and spend their free time has changed over the last forty years. 
Traditional games used to be a very popular way to spend free time, but computer games have become much more 
common in how people spend free time. Advances in technology and modern methods of communication have 
increased accessibility to video games and have made them more commonplace in society (Connolly, Boyle, 
MacArthur, Hainey, & Boyle, 2012). The online video game industry has exploded into one of the strongest and 
most cutting-edge sectors in the U.S. economy (Gallagher 2012). In addition, networked games create the possibility 
of participants playing others from around the globe. Games themselves have also advanced from simplistic games 
like Pac-Man, played by individuals, to extremely complex games involving advanced levels of reasoning and social 
networking involving thousands of players at multi-levels, such as World of Warcraft (WoW ) (Steinkuehler & 
Duncan, 2008). 

The implications for education have become more and more relevant, as games have evolved from simple 
individual games solely for entertainment to participatory learning environments and the application of this learning 
to socially, constructed solutions. Educators seek to update the technology used in school settings, and make 
learning more relevant to today’s world. They also want to move from a teacher-centered orientation to a more 
student-centered orientation. They desire strategies to increase motivation for the enhancement of specific subject 
fields. According to Steinkuehler & Duncan (2008), these benefits can be obtained by studying the use of complex 
games in education. 

Video games have evolved into more complex forms such as massively multiplayer online games (MMOG) 
and massively multiplayer online roleplaying games (MMORPG) (Connolly, 2012). These massive games have 
garnered the attention not just of players but also software developers, researchers, the media, and corporate 
interests. These games are played by hundreds, thousands, even millions of players simultaneously from all over the 
world. They are constantly evolving, both with respect to players’ iconoclastic performance and to the intricacy of 
these players’ social networks. Some researchers investigate whether theoretical formulations from the real world 
are valid within the virtual worlds of MMOGs (Putzke, Fischbach, Schoder, & Gloor, 2010). Theoretical models of 
social selection and influence have been studied to. Findings are still inconclusive, but suggest that there may not be 
strong validity to generalize from real world to virtual world (Putzke et al., 2010). 

The estimated number of players worldwide in MMORPGs is 20 million, and more than half of this 
number is accounted by WoW players. (Billieux, 2012) The numbers show the WoW is the most popular videogame 
and the players are highly motivated to play it. Billieux stated that WoW has reached such success due to the fact 
that players are able to demonstrate and improve upon cognitive and motor skills while engaged in multilevel 
problem-solving challenges. Built into the game are rewards for succeeding on missions or quests that further 
strengthen the skill sets of players (Billieux, 2012) 

One of the reasons for specifically examining the online game (WoW) is that it provides a rich 
environment for investigating how learning takes place in a game and to may enhance learning in educational 
settings. There are several ways that WoW affects learning. First, there are general cognitive, social, and behavioral 
skills that can influence learning regardless of specific academic subjects. (Nardi,2012) Examples of these include 
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learning to communicate, especially across cultures and countries, learning positive behavioral skills as cooperation 
and collaboration, and cognitive skills as problem-solving and logical thinking. Applying WoW to education is 
supported Demski (2009), who discussed a specific component within the same called the Cognitive Dissonance 
Guild. This particular guild was established by an assistant superintendent of curriculum and instruction, and serves 
as a forum for educators to engage other educators, learn from each other, establish independent networking links, 
and explore how WoW and related games can be harnessed for both general and specific school tasks. 

WoW has contributed to raising the level of communication between real life friends, coworkers, and 
family. WoW involves relationships that take place online, but they also extend to include real life interactions. 
WoW is characterized by great attention to social interaction and collaboration like other social-networking sites 
such as Facebook (Williams, Ducheneaut, Xiong, Zhang, Yee, & Nickell. 2006; Snodgrass, Lacy, Dengah II, & 
Fagan, 2011). Players are keen to share experiences, express different opinions, provide assistance, mutually support 
each other, and often talk about their children, parents, sports, or movies (Williams et al., 2006) 

It is the purpose of this research to investigate how school performance can be enhanced by inclusion of 
WoW in an educational setting. This study will specifically address the issue of generalization of skills learned and 
practiced in WoW to educational tasks. 

Players in WoW communicate in real time as they work together to tackle tasks and hopefully achieve 
more efficient results (Thomas, 2009). The environment around the game illustrates the creation and evolutionary 
process of a complex social network, capable of self- improvement by the elimination of older, outdated or useless 
information while adapting to newer content. According to Thomas (2009), one feature of this game is how it serves 
as a scalable learning environment. In WoW, players advance from one level to another gaining in- game skills. The 
longevity of WoW is attributed to this feature, reducing boredom and disinterest because of the high level of 
complexity and the opportunities to continue playing at different levels with increased sophistication and 
specialization. According to Thomas (2009), WoW’s social network is much too large to permit meaningful 
understandings by a single individual or small group. It encourages the formation of larger groups and peer-to-peer 
interactions. He also believes these social networks are much more efficient and effective at formulating answers to 
complex problems. An additional gain is the generation of evolving learning environments, which are able to shape 
and reshape both the context and the content of learning. A final feature of WoW also differentiates it from other 
large scale peer- to -peer social networks. According to Thomas (2009), WoW is the only current game that 
emulates the real world with respect to a climate of constant change, rapidity of assimilating new information, and 
effective and efficient information economy. WoW illustrates not only the evolution of continually more complex 
learning environments but also the generation of new and relevant questions and solutions embedded within a 
context of fast-paced and continuous change. McCreery, Schrader, & Krach (2011), have indicated that evaluating 
21 st century skills are an important point for learning within virtual spaces. They found significant differences 
between expert and novice behaviors in terms of interaction inside WoW. Novices maintained a very low level of 
socio-spatial interactivity during the game. Novices also lacked the basic skills need for negotiating the graphical 
user interface (GUI) for social purposes. On the other hand, experts had the additional skill sets in using the GUI 
and game systems. Oliver and Carr (2009) conducted a study with Virtual Worlds in order to observe how people 
learn from play. The focus of their study was couples that play WoW together. They found players’ learning 
centered around three main categories: ludic, social, and material. The results indicated there was a very simple 
direct relationship between formal education and ludic skills knowledge that are developed and built during play. 
The social and material elements, often associated with problems of distance learning, include the inability to 
achieve a balance between the requirements of learning to play and life obligations. Lack of time also creates the 
need to manage rival demands. 


Dickey (20 1 1 ) has looked at the dynamics of play as an important factor that could impact the learning 
environment. He used WoW in a game design course to examine the impact of game culture and play on class 
dynamics. Results from her study showed that the culture of WoW influenced both student-to-student and class 
dynamics. The study considered peer mentoring, interchangeable roles, fellowship, and cooperation as part of the 
impact on student-to- student dynamics. Dickey noted that peer mentoring and the ability to have interchangeable 
roles were only part of the gameplay, and not present in the classroom setting. On the other hand, both fellowship 
and cooperation were integrated into the classroom environment. 
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Methodology 


For this research into the educational benefits that can be gained from playing a massively multiplayer 
online roleplaying game, World of Warcraft will provide the context. WoW is a worldwide online game that has 
been in play since 2004, and currently has a subscriber base of over 12 million players, continuing to increase in 
numbers of players. As such, it is the most popular MMORP game, with the greatest share of the global market. 
This game takes place in a fantasy world, and at the current time there are nine classes of players who wander 
through this fantasy environment engaged in activities such as hunting, gathering, battling, questing, and crafting on 
their way to increase the strength and progress to more advanced levels. This virtual world allows players from all 
over the world to connect with one another on one playing field, as they engage with one another in these various 
activities. Each new level adds complexity in new locales, with increasingly difficult challenges and tasks to face. 

Data used in this research consists of quantitative data. The data was collected in 2013. The focus of this 
data included threaded discussions of WoW in online discussion forums. During the data collection we used the 
official World of World website (http://fomms.worldofwarcraft.com) to collect our data. The main attention of this 
data included only educational threads that investigate what WoW players write about schooling. The source of our 
data was based on WoW players’ writing about education. In this study, the analysis of WoW players’ writing via 
threaded discussion served as a guide to answers to the research question that asked specifically about the skills 
learned in WoW and how generalize to academic learning tasks. Data from the discussion forums were saved as text 
files. 

We began our study of WoW by opening Safari in Mac and typing the term “Education” in the search box 
in the official World of Warcraft website (http//forums. worldofwarcraft.com). We kept all the information by 
saving it in separate folders so we could access these folders as we needed. Each page was saved by a unique 
number as P09M04 (P09 means (page 9 from (1-100), and M04 means (message 4 in page 9). To save each page 
we did the following: (Control- Right click save as web Archive). We moved from one page to the next page by 
changing the number of the page on URL of the page variable. 

The data analyzed for this particular study consisted of threaded discussions that took place in a 7-month 
period from 8/8/12 (8:51PM) for the first post to 2/22/13 (3:22PM) for the last post. We got 100 pages with 10 
messages. In this investigation, we analyzed a random sample of nearly one thousand discussion posts in which 
participants discussed education. We clicked (command -F) to search about education on each page. 

In this study we considered education as the main distinguishing word to collect the data. To recognize 
common codes from the data collection, which can lead to categories the data, the main goal from this analysis was 
to find which characteristics WoW players shared in the forum under education threads. In order to do this, the 
researchers first began with coding the data collection from the forum. The researchers developed many codes 
related to the collected data. The suggestions for names under the education posts were: Builds on others’ ideas 
(participants restated or summarized accurately what others have said, or asked for clarification or elaboration), 
References to outside resources (references to outside resources were indicated by the tag on the post such as links 
to other threads or articles). No relevant to Education reason (social chat, non-education topics, etc.), and Uncodable 
(cannot tell if it is education related or not). 

The set of codes were developed in order to characterize the WoW-specific content discussion in each 
post. For each post, collection of the content related to Education information included character level, race, class, 
guild status, and player vs. player rank. Moreover, the researcher coded the number of posts per individual thread. 

Results 

The full set of analytic categories used to assess our topic are illustrated by Table 1. 


Categories 

Description 

WoW Game Education 

Occurrences of “education” that referred to knowledge of 
the game environment or learning videogame skills 
specific to WoW were put into this category. 

“Strats must be read if joining our raiding team. 
Understanding and education is vital to a successful raid 
and having each and every person understanding the 
fight before it goes ahead is magical :)” 

“What is the Nether Brigade? The Nether Brigade is a 
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small, secretive and loosely-based military group that 
consists primarily of Warlocks. We do, however, accept 
mages and shadow priests, with the understanding that 
our primary purpose is the education and training of 
those capable of wielding magic of any kind, under 
constant supervision to limit the corruption that is 
characteristic of Arcane and Fel.” 

Education of WoW Characters 

Occurrences of “education” that referred to the fictional 
educational level of a virtual character, acquired game 
skills of the character, and like discussions were put into 
this category. 

“Lisenne was a priestess of the Light training at Tarren 
Mill, because her father thought it was important to 
strengthen quel'dorei relations with the humans and give 
his daughter a traditional education in the virtues of the 
Light, since she wasn't really marriageable anyway.” 

“Concerning other cultures, such as gnomes, goblins, 
blood elves, etc, they may contain high education 
institutes, gnomes especially. Someone correct me if I'm 
wrong, but 1 think Mekkatorque and the evil gnome 
whose name escapes me went to college together.” 

“Real World” Education and 
Playing WoW 

Occurrences of “education” in which forum members 
discussed implications of WoW in relationship to actual 
educational experiences and organizations were put into 
this category. 

“Real life pursuits such as work or pursuing education do 
not in any way equate to something like wow that is done 
purely for entertainment. Finishing a doctorate is an 
accomplishment, downing the latest raid boss in WoW 
really is not, it has absolutely zero significance in the real 
life world and contributes nothing toward paying your 
bills or expanding your mind. All it provides the first 
time or two is some personal sense of satisfaction but 
after that it is on farm and no longer matters anymore.” 

“With teaching anything (not just language), using a 
medium that is attractive or entertaining has a lot of 
potential. This can be done through music, 
television/movies, books/magazines/comics, video 
games, etc. We remember the things that stand out easier 
or that draw us in. The fact we can potentially immerse 
ourselves and develop our own motivation to learn more 
is a very powerful tool to use in education.” 

“Real World” Education 

Occurrences of “education” in which forum members 
discussed actual educational experiences and 
organizations with no connection to WoW or 
videogames were put into this category. 

“You're not buying a Yugo. Look at any sort of 
certifications you may need, and talk to your school 
counselor to make sure that lines up with your plans. 
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Also, if you plan on going for a masters or doctorate, 
research which majors and fields of study they accept. 

Do the same for the kinds of jobs you thing you would 
be interested in, to get a good idea of what kind of 
education you need.” 

“This probably seems really offensive, and I apologize 
for the generalizations I've made, but the truth is that 
teachers don't have the time or resources to give an 
individualized education to every single student, every 
day, especially in states such as mine where students 
with special needs are mainstreamed into classrooms 
with students without such needs. Currently my wife has 
a class with several IEPs (education plans for 
accomodations for students with special education needs) 
and two students with behavior issues.” 

Jokes and Insults about Education 

Occurrences of “education” in which forum members 
referred to someone’s educational level in a joking or 
insulting manner were put into this category. 

“Secondly, I don't care what education you've had, or 
how well you did in those courses. As far as I am 
concerned, you are just another person on the other side 
of the screen, e-raging over something that is a very 
simple, and very easy concept to grasp.” 

“I am sure this passes for clever in the education system 
you came out of.” 

Other 

Occurrences of “education” that could not be readily 
placed in one of the previous five categories were 
assigned to the “other” category (and are still being 
reviewed). 


The results showing percentages of responses in each category are illustrated by Figure 1. 
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Our goal has been to examine World of Warcraft player’s attitudes and beliefs about education and 
schooling through discussions in online game forums. This study aimed to answer this research question: what do 
WoW players write about schooling in relation to discussions about WoW in online discussion forum. 

The data. We have created six categories: jokes and insults about education, real world education, other, WoW game 
education, real world education and playing WoW, and education of WoW characters. 

Overall, the results are surprising; the data showed that the most comments in the thread discussions of WOW 
educational forums are related to jokes and insults about education. The results demonstrated that the forum 
members usually have fewer discussions about education of WoW characters and real world education and playing 
WoW. 

From the category percentage schedule, we will discuss the category from the highest percentage to the lowest. 

1 . Jokes and Insults about Education 

This category had the highest percentage in this study (29.3%). The forum members mentioned jokes 
and insults to others’ level of education. For example, the insults included insults about the US 
education public school systems, insults directed at another player’s intelligence, and making fun of 
another player’s post such as “I know reading comprehension in the US isn’t so great since the 
education system sucks”. 

2. “Real World” Education 

This is a second highest category by (22.5%). The forum members discussed the real education 
experiences and organization that are not related to the game WoW. The members discussed such 
topics as; comparing knowledge and experience, discussing character and skill education, and players 
giving advice to another about advancing their real world education. For example “if you remain 
interested in this topic as your education advances, you should look into operation research as an area 
of advances study. You will need an undergraduate degree in math to apply for most graduate 
programs”. 
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3. Other 

This category had (21.7%) which included comments that didn’t fit in the other five categories we 
were assigned and still being reviewed. 

4. WoW Game Education 

The members in this category had (13.8%), they discussed the knowledge of the game environment or 
learning videogame skills specific to WoW. The topics mentioned in this category included discussing 
other players not knowing enough about an aspect of the game, discussing gaining more knowledge 
about something in the game, and discussing creation of a game tutoring or educational system: for 
example, “it needs more exposure I think, and more education in how it works in a PvP scenario”. 

5. “Real World” Education and Playing WoW 

In this category, the percentage was low (7.4%). The data was surprising because we expected more 
discussion and support this category, which is one of the foci of our study. The members discussed 
such topics as: discussing how people need to be more educated in real life in order to understand all of 
the references to real life they need for the game, opinions about children playing the game and people 
needing an education, and discussion of things found in the game that come from real life like Chinese 
history: for example, “there are also very subtle reference to art works, literature, movies, even poetry 
that you need to have a profound culture and education to note”. 

6. Education of WoW characters 

This is the lowest percentage in the categories: it had (5.3%). This category referred to the fictional 
educational level of virtual character, acquired game skills of the character, and similar discussions. 

The topics discussed in this category were talking about a character and place in the game, ideas about 
the race of the characters, and discussion of the role-playing character’s biography: for example, your 
father's loyalty to the King provided you with a great education and allowed you to build a friendship 
with Prince Varian. Both of your parents were killed when Garona Halforcen betrayed the King, and 
led the Horde to raise Stormwind and it's land”. 

In summary, this study examined WoW because it provides such a rich environment for 
investigating how learning takes place in a game and how it may enhance learning in educational 
settings. Analysis of the categories above yielded negative results. The members in this study had a 
negative attitude toward education. When we compared between highest and lowest percentages, we 
found the highest percentage was related to jokes and insults about education. This is a troubling 
commentary on the educational system. 
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Abstract 

The combination of instructional multimedia (IM) and college students’ characteristics may have an effect 
on the amount of effort one puts forth to understand information. This quantitative study explored the relationship 
between different forms of IM and college students’ epistemological learning beliefs (ELBs) impact on perceived 
mental effort (PME) in an undergraduate asynchronous course. College students (n=182) were randomized into three 
IM groups: audio with text and graphics (ATG), text with graphics (TG), and video with audio, text, and graphics 
(VATG). Students in Group 1: ATG and Group 2: TG reported lower PME than those in Group 3: VATG (p < 0.05). 
In Group 1: ATG, students with naive ELB in the structure of knowledge dimension reported lower PME scores 
after viewing the supplemental materials (p<0.05). In Group 2: TG, students with naive ELB in structure of 
knowledge and speed of knowledge acquisition dimensions reported higher PME scores (p<0.05). 

Introduction 

As online learning continues to grow, attention has been directed to understanding students’ cognitive 
learning processes within this environment. Examples include the evaluation of students’ success in online learning 
environments and the relationship of instructional multimedia (IM) and cognition in an online environment (Florax 
& Ploetzner, 2010; Ibrahim, Antonenko, Greenwood, & Wheeler, 2011) and the role of epistemological learning 
beliefs (ELBs) and cognition (Barnard, Lan, Crooks, and Paton, 2008; Braten, Stromso, and Samuelstuen, 2008). 

The present study uses Cognitive Theory of Multimedia Learning (CTML) and Schommer’s ELBs to understand the 
relationship between IM and ELBs effect on perceived mental effort (PME) in asynchronous environments as few 
studies have explored this notion. 


Cognitive Theory of Multimedia Learning 

Mayer formulated CTML to understand how the design of IM can improve learners’ comprehension in 
technology based environments (Mayer, 2010). The instructional design of IM can impose three types of cognitive 
load: intrinsic, extraneous, and germane, which impacts learning (Mayer, 2010; Mayer & Moreno, 2003; Sweller, 
2010). The complexity of the task and the level of a learners’ expertise to the subject matter determines intrinsic 
cognitive load (ICL). The instructional features that are not beneficial for learning contribute to extraneous cognitive 
load (ECL). The instructional features that are beneficial to learning contribute to germane cognitive load (GCL) 
(Leppink, Paas, Van der Vleuten, Van Gog, & Van Merrienboer, 2013). CTML draws on the following theories: 
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Paivio’s (1986) dual coding theory, Baddely’s (1992) model of working memory, Sweller’s (1988) and Chandler 
and Sweller’s (1991) CLT, Wittrock’s (1989) generative theory, and Mayer’s (1996) select, organize and integrate 
(SOI) model of meaningful theory (Mayer & Moreno, 1999). CTML is based on three assumptions from these 
theories: dual channels, limited capacity, and active processing (Mayer, 2010). The dual-channel assumption is 
based on Baddeley’s (1992) and Paivio’s (1986) work, which proposes students have separate channels for 
processing verbal and visual material. The second assumption, limited capacity, is based on Baddeley’s (1992) and 
Chandler and Sweller’s (1991) work and proposes students can process only a few elements in each channel at any 
one time (Mayer & Moreno, 2003). The final assumption, active processing, is based on Wittrock’s (1989) and 
Mayer’s (1996) theories and proposes meaningful learning occurs when “students engage in appropriate cognitive 
processing during learning, including attending to relevant material, mentally organizing it into a coherent cognitive 
representation, and integrating it with prior knowledge activated from long term memory” (Mayer, 2010, p. 544). In 
an effort to design material that reduces ECL, Mayer proposed nine methods: off-loading, segmenting, pre-training, 
weeding, signaling, aligning, eliminating redundancy, synchronizing, and individualizing. For this study, weeding 
and segmenting were used to design the IM (Mayer, 2010; Mayer & Moreno, 2003) as prior indicated designing 
materials based on these methods improved comprehension of the subject material (Florax & Ploetzner, 2010; 
Ibrahim et al., 2011; Mayer, 2010; Mayer & Moreno, 2003). 

The weeding method claims when unnecessary graphics are removed, deeper learning occurs, which creates 
a coherence effect (Mayer, 2010; Mayer & Moreno, 2003). Ibrahim et al. (2011) examined students with low prior 
knowledge to the subject material and the impact the weeding method has on their cognitive load with educational 
videos. They concluded the weeding method improved learners’ comprehension by reducing ECL. On the other 
hand the weeding method does not always have an impact on comprehension as explained by De Westelinck, 

Valcke, De Craene, and Kirschner (2005). They discovered students, with low prior knowledge to the subject 
material, graphics used to explain the information increased mental effort and resulted in poor performance on the 
learning task. They concluded the higher PME scores may have been correlated to ECL, which is the reason students 
did not perform well on the learning task, which limited learning that took place. 

The segmenting method claims students will learn the information better when they can control the content, 
i.e. stop, pause, and start, rather than view the presentation continuously, which creates a segmenting effect (Mayer 
& Moreno, 2003). Florax and Ploetzner (2010) conducted a study in which students with low prior knowledge to the 
subject material were placed into two groups: segmented and continuous to view text-based material. They 
determined students in the segmented group performed better than students in the continuous group. They concluded 
the segmented group had lower amounts of ECL than the continuous group, which led to better performance and 
learning. 


Epistemological Learning Beliefs (ELBs) 

ELBs can be defined as, “students’ beliefs about the nature of knowledge and learning” (Schommer-Aikins, 
Duell, & Barker, 2003, p. 347). Schommer (1990) suggested ELBs are not inborn, but evolve with time. She devised 
a framework that comprises five dimensions of ELBs: the structure (simple vs. complex), certainty (certain vs. 
tentative), the source of knowledge (omniscience vs. personal construction), control of knowledge acquisition (fixed 
vs. malleable) and speed of knowledge acquisition (quick vs. gradual learning). Structure, certainty, and source of 
knowledge dimensions were based on Perry’s (1968) work. He noted once students move through their college 
years, their knowledge matures, and thus, learning is considered as complex, tentative, and under their personal 
authority. Control of knowledge acquisition dimension was based on Dweck’s (1988) work. He notes that students 
believe intelligence is fixed or incremental. Speed of knowledge acquisition dimension was based on work of 
Schoenfeld (1983, 1985). He indicates students believe knowledge can be acquired quickly in an all-or-none format. 
These five dimensions can be further categorized into either naive or sophisticated ELBs. A naive ELB would be 
students believing if they cannot leam the information quickly, they will never learn it. Whereas a sophisticated ELB 
would be students believing learning acquisition is a gradual process. It is important to note that a student does not 
have to be naive or sophisticated in all five dimensions, as each dimension is independent of one another (Oh & 
Jonassen, 2006; Schommer, 1990; Schraw, Bendixen, & Dunkle, 2002). 

To gauge the level and direction of ELBs, Schommer (1990) developed a 63 item questionnaire that 
assesses each of these five dimensions referred to as Schommer Epistemological Questionnaire (SEQ). There are 
concerns, however, with her questionnaire such as the scoring procedure, psychometric properties and reliability 
(Clarebout, Elen, Luyten, & Bamps, 2001; DeBacker, Crowson, Beesley, Thoma, & Hestevold, 2008). To address 
those measurement issues, Schraw et al. (2002) developed a reduced version of Schommer’s questionnaire referred 
to as the epistemic belief inventory (EBI). Barnard et al. (2008) determined students enrolled in an online 
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environment that have sophisticated ELBs performed well in an online environment. They concluded that if 
learners’ ELBs are sophisticated (i.e. disagree with the statements concerning simple knowledge), learning 
increases, which may account for better performance in an online environment. Although, Braten et al. (2008) in 
their web-based study found mixed results on the association between sophisticated and naive ELBs and students’ 
cognitive processing. Students who held sophisticated ELBs towards the structure dimension (i.e., knowledge is 
complex) along with naive ELBs towards the source (i.e., omniscient authority) had a better cognitive ability to 
comprehend the information than those who held naive ELBs about the structure (i.e., simple knowledge) and 
sophisticated ELBs about the source (i.e., personal authority). They concluded that ELBs should not be assumed to 
be universally effective or adaptive. This means that learning and comprehension depends on the context, design, 
and the students’ level of expertise with the subject material. 

Measuring Mental Effort 

Properly measuring the cognitive load affecting student comprehension and performance would help 
instructional designers and researchers understand the type of IM to use for particular environments, i.e. online, and 
learners (Leppink et al., 2013). Although, measuring cognitive load is difficult since, depending on the situation, 
many factors impact learning performance such as mental effort, which considers subject characteristics and 
students’ learning environments. Considering mental effort encompasses many factors, it is a better way to 
determine the relationship between IM and ELBs effect on PME. Paas (1992) created a rating scale referred to as 
perceived mental effort (PME) to determine the amount of mental effort one puts forth to understand information. 
The scores on this instrument correspond to overall cognitive load (Gimino, 2002; Van Gog & Paas, 2008), but 
cannot distinguish which type of cognitive load is causing the low or high scores. Particular studies have used the 
PME scale to determine the impact IM and/or ELBs have on one’s comprehension or performance in controlled 
environments (Chien & Chang, 2012; Lin, 2002; Medula, 2012; Scheiter, Gerjets, Vollmann, and Catrambone, 
2009). Chien and Chang’s (2012) study reported that using audio or video IM to design and deliver science courses 
asynchronously among college students reduced their perceived mental effort (PME), which correlated to a higher 
performance within the course. Medula’s (2012) study, though, discovered overreliance on IM caused students to 
expend more mental effort, thus performance was low. He separated novice college students into three instructional 
multimedia groups: text, audio with text, and video with audio and text, and then recorded their PME scores. He 
reported that mental effort and potentially ECL was higher among the audio with text group and video with audio 
and text group due to the over stimulation of the IM. Lin (2002) conducted a study in a web-based controlled setting 
to determine if video and the type of ELBs led to high or low perceived mental effort. He determined that those 
students, who have strong naive ELBs in the source of knowledge dimension, reported higher PME scores. Scheiter 
et al. (2009) evaluated the use of video IM and its effect on student PME within a web-based controlled setting. 

They showed that students with naive ELBs in the dimensions structure of knowledge, certainty of knowledge, and 
speed of knowledge acquisition reported higher PME scores. 

Based on the studies presented, few have determined a relationship between IM and college students’ ELBs 
and the effect on PME scores within an asynchronous environment. Research questions were formulated to address 
the gaps in literature within this study: 

1 . Which type of instructional multimedia impacts perceived mental effort? 

2. What is the relationship among learning beliefs and IM groups that impacts perceived mental effort? 

Method 

Setting 

The study took place in spring 2012 in an undergraduate general education course at a large land grant 
University in the upper Midwest. The general education courses generally have a large enrollment (i.e., >100 
students), a diverse amount of students from different majors and grade levels (i.e., freshmen- senior), and a variety 
of age groups (i.e., 18-22 years old). Therefore, a course within the University which fulfills one of the general 
education requirements within natural sciences and technology and meets the above characteristics of a general 
education course is within the food science and human nutrition (FSHN) department: FSHN 101- Introduction to 
FSHN. The FSHN 101 course also has students complete many of the activities asynchronously through Compass 
2g (the University’s learning management system). Students are informed about Compass 2g in their first lecture. 
This course introduces students to the basic concepts of food science and human nutrition that combines 
introductory mathematical and science concepts. The course is divided into four content sections: unit 1: nutrition 
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and health; unit 2: food composition and chemistry; unit 3: food microbiology and processing; and unit 4: food laws, 
quality, and the consumer. 


Participants 

The study utilized convenience sampling due to the research team’s access to this student population. A 
total population of 1 82 college students completed all facets of the study. Students were randomized into one of 
three IM groups based on their pre-test scores: Group 1- audio + text + graphics (ATG), Group 2- text + graphics 
(TG), and Group 3- video + audio + text + graphics (VATG). The sample was comprised of significantly more 
female (76%) than male (24%) students and represented all college years: freshmen (45%), sophomores (26%), 
juniors (15%) and seniors (14%). Also, the majority of students taking the course were from non-science majors 
(English, pre-law, education) (80%) compared to science majors (chemistry, biology) (20%) (see Table 1). 

Table 1 

Participant Characteristics 


Participants 

(n=182) 

Group 1-ATG 
(/?= 60) 

Group 2-TG 
(n= 61) 

Group 3-VATG 
(n= 61) 

Total 

(%) 

Gender 

Male 

16 

14 

14 

24 

Female 

44 

47 

47 

76 

Year in School 

Freshman 

29 

22 

31 

45 

Sophomore 

17 

18 

13 

26 

Junior 

9 

12 

6 

15 

Senior 

5 

9 

11 

14 

Major 

Science 

13 

11 

12 

20 

Non-Science 

47 

50 

49 

80 


Materials 

The food science materials consisted of three, two and a half minute segments within unit 4: food law, 
quality assurance and sensory tests. These topics were chosen as historically students performed poorly on the 
exams per the instructor. Although the content in each of the materials remained the same for each IM group, the 
delivery mode was different. For Group 1-ATG, the instructor recorded voice over using Microsoft Expression 
Encoder v. 4 Screen Capture. Expression Encoder is an audio recording device and once recorded the voice 
recording can be embedded within any system or program (Microsoft, 2013). For Group 2-TG, the instructor 
designed the food science material using MS PowerPoint. For Group 3-VATG, the instructor embedded herself 
within the food science material using Personify (previously known as Nuvixa StagePresence v. 1.1.1). Personify 
uses a Kinect system (i.e. motion detector) created by Microsoft and once downloading the software allows the 
presenter to record themselves in front of PowerPoint (Microsoft Office 2010), documents, or web screens to help 
explain the information (Personifyinc, 2013). Graphics used for all three IM groups were pictures, diagrams, and 
graphs to explain the material. The texts for Groups 1- ATG and 3- VATG consisted of short words to highlight the 
audio information. The texts for group 2-TG consisted of lengthier sentences on each slide (see Figure 1). 
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Figure 1. Representation of the three instructional multimedia groups with quality assurance content. 
Reprinted with permission. 


Measures 

To determine students’ prior knowledge, a pre-test was provided to students before they viewed the food 
science materials. The pre-test included six multiple choice questions (e.g., which of the following is an example of 
using the process control model for quality control?) and one true/false question (e.g., the statistically based 
approach to simultaneous quality and productivity improvement developed by Dr. W. Edwards Deming was readily 
accepted by the United States). The pre-test had a maximum point value of 20. Students who scored less than 70% 
were considered low prior knowledge as per the decision of the instructor. Based on the pre-test, 91% of the students 
scored below 69% and 9% of the students scored above 70%, which indicates a majority of the students had low 
prior knowledge to the food science materials. 

To determine students’ ELBs, Schraw et al.’s (2002) EBI was provided to students before they viewed the 
food science materials. The EBI contains 32 questions, which are based on five-point Likert scale (l=strongly 
disagree to 5=strongly agree) (Schraw et al., 2002). Students subjectively rate their beliefs on a variety of questions 
that determine their naivety or sophistication based on the five dimensions (e.g., too many theories just complicate 
things- assessing structure of knowledge dimension). Subjects who “strongly disagree” with scale’s dimensions are 
considered to have sophisticated ELBs. While those students who “strongly agree” with scale’s dimensions are 
considered to have naive ELBs. Hardre, Crowson, Ly, and Xie (2007) administered the EBI in a web-based format 
and after testing the internal consistency of the five dimensions, they found the Cronbach alpha’s ranged from 0.43 
(structure of knowledge) to 0.79 (speed of knowledge acquisition). 

To determine students’ perceived mental effort, Paas’ (1992) PME scale was provided once students 
viewed the food science materials. Students subjectively rated their PME on a nine-point Liket scale after each food 
science material. Students were asked to subjectively estimate their mental effort based on two questions “In study 
the lesson I invested” and “How much mental effort did you put forth to understand the lesson?” with ratings 
ranging from “very, very low” to “very, very high”. 


Procedure 

The study was conducted over a two week time span due to the design of the actual course, although the 
time to complete the activities within one sitting would have been approximately two hours. Students had the 
opportunity to win a $50.00 gift card after completion of the study. First, students completed the pre-test and the 
EBI. They then viewed the three food science materials at specific points in the course. They completed the PME 
scale after viewing each food science material, thus completed a total of four PME scales. 

Data Analysis 

Data is presented as frequencies, percentages, means, and standard deviations. An alpha level of 0.05 for all 
statistical tests was deemed significant. Validation and identification of reliable constructs from the EBI data was 
necessary. After students finished the 32-item EBI, all items on the questionnaire were first examined for reliability 
using exploratory factor analysis with Varimax rotation to extract factors (Matsunaga, 2010). Based on the Kaiser 
rule (Kaiser, 1960), factors with eigenvalues greater than 1.0 were considered. Only dimensions and questions that 
had a factor loading of 0.5 and above were retained (Ledesma & Valero-Mora, 2007). Parallel analysis was then 
conducted to verify the findings of factor analysis since applying the K1 rule to extract factors might result in over 
or underestimation of these factors (Ledesma & Valero-Mora, 2007). This study had IM as the independent factor 
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with three groups: ATG, TG, and VATG. Each of the EBI dimensions were also considered independent variables 
and used as such within correlation analyses. The dependent variable in this study was the changes in PME scores. A 
one-way AN OVA was used to identify factors effects of IM on PME scores. Mean Fisher least significant difference 
(LSD) post hoc analyses were applied after finding significant factors effect. A correlation analysis was employed to 
determine relationship between EBI scores and PME scores within IM groups. 

Results 


Data Reduction and Reliability of EBI 

The initial factor analysis extracted five factors with eigenvalues greater than 1.0. After the parallel 
analysis, it resulted in the retention of seven items in two factors. Factor 1 was labeled structure of knowledge (SK), 
which contained four items and their corresponding factor loadings. Factor 2 was labeled speed of learning 
acquisition (SLA), which contained three items and their corresponding factor loadings (see Table 2). The internal 
reliabilities of the two factors (SK and SLA) were acceptable at Cronbach alphas of 0.6 and 0.74, respectively 
(Tuckman, 1999). 

Table 2 

Factor Loadings for EBI 


Item Factor Loading 

1 (SK). Too many theories just complicate things 0.80 

2 (SK). The best ideas are often the most simple. 0.66 

3 (SLA). If you don't learn something quickly, you won't ever learn it. 0.69 

4 (SK). Things are simpler than most professors would have you believe. 0.54 

5 (SLA). If you haven't understood a chapter the first time through, going back 0.72 

over it won't help. 

6 (SLA). Working on a problem with no quick solution is a waste of time. 0.71 

7 (SK). You can study something for years and still not really understand it. 0.54 


Question 1: Which type of IM impacts PME scores? 

Table 3 describes the means and SDs for PME scores for each of the IM groups. The type of instructional 
multimedia (main effect) explained changes in PME scores (ANOVA; F [2, 179] = 3.64, p = 0.02). Fisher LSD post 
hoc mean analyzes showed that students in Group 2: TG reported on average lower perceived mental effort scores 
than Group 2: ATG and Group 3: VATG (p < 0.05) (see Table 3). 

Table 3 

One-Way ANOVA for the type of IM that impacts PME scores 


Source 

P 

SD 

df 

SS 

MS 

F 

P 

IM Groups 



2 

9.97 

4.99 

3.64 

0.03* 

ATG 

5.05 

1.25 





0.72 

TG 

4.58 

1.28 





0.03* 

VATG 

5.10 

0.95 





0.82 

Error 



179 

245.14 

1.37 




Note. *p<0.05 


Question 2: What is the relationship among learning beliefs and IM groups that impacts perceived mental effort? 

A Pearson correlation analysis was conducted to determine if there was a relationship between ELBs and 
PME scores within each of the IM groups. Among students in Group 1 : ATG, there was a negative weak correlation 
between PME scores and structure of knowledge ELB dimension (r = -0. 1 4. p = 0.03). Among students in Group 2: 
TG, there was a positive, weak correlation between PME scores and structure of knowledge ELB dimension (r = 
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0. 11, p = 0.01) and speed of knowledge acquisition ELB dimension (r = 0.15, p = 0.01). There were no correlations 
in Group 3: VATG (see Table 4). 

Table 4 

Pearson Correlation Analysis among ELBs, IM Groups, and PME scores 


ELBs 

Group 1 

: ATG 

Group 2: TG 

Group 3: VATG 


r 

/7-value 

r 

/7-value 

r /7-value 

Structure of knowledge 

-0.14 

0.03 

0.17 

0.01* 

-0.09 0.12 

Speed of knowledge acquisition 

0 

0.48 

0.18 

0.01* 

-0.07 0.18 


Note. */7<0.05 


Discussion and Conclusion 

In sum, the results reveal there is a relationship among IM and college students’ ELBs, which has an effect 
on one’s PME. Results from research question 1 indicated students within Group 2: TG reported lower PME scores 
than those in the other IM groups. Based on the results from our study, it is possible that these higher PME scores 
reported by students in Group 1: ATG and Group 3: VATG, however relatively small, could be due to the overuse of 
IM within these supplemental materials. These findings were consistent with Medula’s (2012) findings which he 
indicated an overstimulation of IM could lead to higher PME scores. In an attempt to reduce any potential 
extraneous load, the food science materials were designed by weeding and segmenting CTML methods. Considering 
the PME scale does not determine which type of cognitive load one was experiencing, the researchers cannot 
conclude if the CTML methods did have an impact on one’s PME scores. If college students were focusing more on 
the design of each of the food science materials, i.e. voice or head of the instructor, the graphics used within the 
materials, then their PME scores might have been more reflective of extraneous cognitive load. Whereas if college 
students were focusing on the content of each of the food science materials, then their PME scores might have been 
more reflective of germane cognitive load. Thus, these methods may have worked and students’ PME scores were 
higher due to attempting to understand the content. Considering this evidence, when designing learning materials 
using IM it is critical to review the content to reduce cognitive overload, but also, understand the characteristics of 
the students that may perceived these materials differently. 

Results from research question 2 indicated students in Group 2: TG reported higher PME scores if they 
held naive ELBs in structure of knowledge and speed of learning acquisition dimensions. This implies if students 
have strong naive ELBs, they will put forth a greater amount of effort to understand the information presented to 
them in text. Few studies have related ELBs, PME and IM, but many have associated naive ELBs with poorer 
performance or lower cognitive processing (Barnard et al., 2008; DeBacker & Crowson, 2006). Barnard et al. (2008) 
reported that students who held naive learning beliefs performed poorly in a controlled online environment. 
DeBacker and Crowson (2006) found a similar relationship when studying cognitive engagement and learning 
beliefs. They concluded that when students believed that acquiring knowledge was simple, their cognitive 
processing was lower. This study did not measure, students’ performance in the course or determine cognitive 
processing, thus no assertion can be made if a higher PME score was associated with higher or lower cognitive 
processing. The results further identified students in Group 1: ATG, who held strong naive ELBs in structure of 
knowledge dimension reported lower PME scores. Thus, students who believe learning is simple may put forth 
lesser amounts of mental effort when reviewing information designed with audio. This may be for the fact 
extraneous graphics are removed and students can hear the information instead of reading it. 

Finally, our results from Group 3: VATG revealed no significant relationship between ELBs and PME. 
Other authors have obtained different results. Lin (2002) determined students, who have strong naive ELBs in 
source of knowledge dimension, reported higher PME scores when viewing instructional videos. Similarly, Scheiter 
et al. (2009) showed that students with naive ELBs in structure of knowledge, certainty of knowledge and speed of 
learning acquisition dimensions reported higher PME scores when viewing video web-based instructional material. 
Although the results of this study do not support these findings, it is possible that these effects might be due to the 
difference in the learning setting (i.e., natural vs. controlled setting), as these authors were able to control several 
confounding factors. 
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Implications for the Design of Asynchronous Online Learning 

This study explores several important points that could be useful for instructional designers. First, the 
findings strengthen current evidence on the potential relationship between college students’ ELBs and IM use in the 
asynchronous online learning setting. The results indicated the design of the particular IMs (ATG and TG) may 
facilitate or limit students’ mental effort depending on their ELBs. Therefore, instructional designers can improve 
the design and content of IM to reduce any potential extraneous cognitive load by considering one’s learning beliefs 
in any type of higher educational setting; i.e. online or face to face. 

Limitations and Conclusion 

Identifying factors that effect PME is critical to understand the impact these scores have on performance 
within the classroom. This study was conducted in a current course that had an asynchronous component and thus, 
controlled factors were limited to IM groups, ELBs and prior knowledge. Factors at large such as students’ learning 
environment (e.g., viewed the materials at home, in class) and time-on-tasks were not controlled, which may have 
also impacted PME scores. For future studies, these would be crucial elements to include. In regards to measuring 
ELBs by the EBI, we are depending on college students to accurately depict their beliefs on a subjective scale. 
College students may have answered each of the EBI questions based on their certain attitudes and memories and 
how they understood and interpreted the question. Future research we conduct in this area should also include 
qualitative data to understand more about one’s ELBs. Additionally, from the limitations of the PME scale we 
cannot concretely determine if these scores were an indication of a particular cognitive load; i.e. extraneous and/or 
germane. Hence, discovering ways to accurately determine cognitive load is necessary. One of which is Leppink et 
al.’s (2013) subjective rating scale that measures the different types of cognitive load. Thus, for future research we 
should consider implementing different subjective scales. Further investigation as to which cognitive load college 
students’ experience when viewing IM designed by the methods of CTML with their ELBs and factors will help 
instructional designers determine if these techniques due contribute to better performance within asynchronous 
environments. 
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As society embraces the use of learning technologies that continue to become a vital part of today’s 
educational experience, institutions scramble to integrate them. Educational professionals, often charged with these 
technology integration efforts, strive to improve the learning experiences and outcomes of students. Such a complex 
and systemic initiative calls for a well-conceived implementation plan that is often absent or insufficient. Thus, an 
interest in diffusion of innovation (DOI) theory as it relates to instructional technology integration has re-emerged. 

DOI theory seeks to understand the social process that community members go through in order to adopt or 
reject an innovation (Rogers, 2003; Surry & Farquhar, 1997) such as a new instructional technology. Donald P. Ely 
focused his research on the implementation phase of DOI (which he posited was the least understood component) 
and successfully identified eight conditions for implementing technology innovations (Surry & Farquhar, 1997). 
These conditions are: dissatisfaction with the status quo; knowledge and skills; availability of resources; availability 
of time; rewards and/or incentives; participation; commitment; and leadership (Ely, 1976, 1999). Given the barriers 
that can arise with the integration of instructional technology within higher education, the study of DOI in relation to 
instructional technology can assist in supporting the integration process (Surry, 2002). Instructional design and 
technology professionals and their clients can benefit from a focused effort to integrate Ely’s eight conditions into 
the implementation of technologies for teaching and learning, particularly in a large, complex context such as the 
university. 

Eleven years ago, a large research university in the United States decided to enact a university-wide 
electronic portfolio (ePortfolio) initiative to enhance student learning, especially in areas such as practice, feedback, 
and assessment. Several credible strategies and resources were consulted to support this initiative’s success 
including: strategically aligning the initiative to department, college, and/or institutional goals and missions; 
fostering partnerships with key stakeholders; running pilots; offering faculty development opportunities; and the 
consultation of the Concerns-Based Adoption Model (CBAM) for change (Hord, Rutherford, Huling-Austin, & Hall, 
2006), and Ely’s eight conditions of implementation. However, it was unclear to what degree these elements were 
systematically and successfully woven throughout the initiative. Thus, a study was initiated to determine the degree 
to which DOI elements informed ePortfolio implementation and, subsequently, develop a framework for 
implementation that infused critical aspects of DOI into a functional tool that could be used by higher education 
professionals pursuing ePortfolio integration at other institutions. 

The study followed Richey and Klein’s (2007) model development methodology, also known as a Type 
Two development research approach, that includes three phases: analysis, development and revision, and evaluation. 
Because DOI was a conceptual frame informing the study, the researchers selected an existing instrument for 
investigating technology integration through the lens of DOI theory. The RIPPLES survey, developed by Dr. Daniel 
Surry and Dr. David C. Ensminger, is based on the combined results of a literature review of DOI theory, a survey 
of college deans’ opinions regarding the factors affecting technology integration, and Dr. Surry’s personal 
experiences with innovation adoption (Surry, 2002, 2005). Permission was sought and granted by Dr. Surry to use, 
and modify as needed, the original RIPPLES survey for use in the study. The purpose of this paper is to revisit the 
original RIPPLES survey, present the modifications made to the survey along with supporting rationale, and reflect 
on the application of the revised survey along with recommendations for future use. 

Original Survey 

The original RIPPLES survey consists of seven elements believed to be critical to the adoption of an 
innovation: resources, infrastructure, people, policies, learning, evaluation, and support (Surry, 2002, 2005). Each of 
the seven elements is explored individually in seven sections of survey through a series of close-ended questions. 
Each section provides a definition of the element being examined in that section and poses four questions asking 
participants to rate: the availability of the element in their organization; their organization’s allocation of the element 


18 



in relation to the innovation; their organization’s allocation of the element as a barrier or enabler for the 
implementation of the innovation; and the importance of the element to the successful use of an innovation. Each of 
these questions had their own rating scale. Scales included: Low to Above Average; Extremely Appropriate to 
Extremely Not Appropriate; Major Barrier to Major Enabler; and Extremely Important to Not Important. The lack of 
consistency across rating scales as well as certain element definitions were of concern when considering application 
to the study ePortfolio implementation study. 

In addition to the seven sections for the RIPPLES elements there were two other sections to the original 
survey. These sections explored participant demographics and opinions. The demographics section asked nine 
questions. This section requested the participant's location, provider (organization) type, gender, age, and primary 
work location. This section also asked for a participant’s personal rating of innovativeness; personal rating of his/her 
use of innovative practices and techniques; a rating of the organization’s overall innovativeness; and a rating of the 
organization’s use of innovative practices and techniques with the specific innovation in question (e.g. web-based 
learning). In the demographics section, the rating scales for close-ended questions once again varied from one 
another. This variation, along with the irrelevance of certain demographic questions, suggested the need for 
additional revisions prior to use in the ePortfolio study. 

The opinion section of the original RIPPLES survey posed four open-ended questions. Participants were 
asked to indicate: the two biggest barriers to innovative practices for a specific innovation’s use (e.g. web-based 
learning); the two biggest enablers to innovative practices for a specific innovation’s use; their approach to fostering 
innovative practices for a specific innovation’s use, and any other information they would like to share. The survey 
closed with an option to provide contact information for follow-up questions. It appeared that this final section 
would require minimal revision for use in the ePortfolio study. 

Revised Survey Before Survey Administration 

As previously mentioned, the researchers intended on exploring the process, and associated successes and 
challenges, in a university’s integration of ePortfolios. Since both the university’s ePortfolio implementation effort 
and the RIPPLES model were both framed, in part, by Ely’s eight conditions of implementation, the RIPPLES 
questionnaire was selected as the survey instrumentation for the study. The researchers compared the elements of 
RIPPLES with Ely’s eight implementation conditions to ensure as close alignment as possible. (See Table 1.) The 
correspondence of these components was found to be as follows: RIPPLES’ resources and infrastructure elements 
were found to correspond with Ely’s condition of availability of resources; RIPPLES’ people element was found to 
correspond with Ely’s condition of participation; RIPPLES’ policies element was found to correspond with Ely’s 
condition of leadership; RIPPLES’ learning element was found to correspond with Ely’s condition of sufficient 
knowledge and skills; RIPPLES’ evaluation element was found to correspond with Ely’s condition of rewards or 
incentives; and RIPPLES’ support element was found to correspond with Ely’s condition of commitment. Two of 
Ely’s conditions of change were not found to be reflected in the original RIPPLES survey. These conditions were 
dissatisfaction with the status quo and availability of time. Thus, while most of the eight conditions were present in 
some form in the original RIPPLES model, greater alignment through a redesign of the survey could result in more 
valid data to inform the ePortfolio study. 

Table 1 

Comparison of RIPPLES Survey Elements to Ely ’s Eight Conditions of Implementation 

RIPPLES Survey Element and Definition Ely’s Corresponding Condition of Change 


Resources - financial resources 

Infrastructure - technological backbone of the 
university 

People - social and human elements 

Policies - written and unwritten rules, practices, 
traditions, and regulations 

Learning - instructional outcomes of training 


Availability of Resources 
Availability of Resources 

Participation 

Leadership 

Sufficient Knowledge and Skills 
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Evaluation - assessment of student goals, technology, Rewards or Incentives 
technology plans, innovative practices, costs^enef^ts 

Support - training, technical support, pedagogical Commitment 
support, administrative leadership 


For better alignment, the original RIPPLES resources element, which only included financial resources, 
was revised to include time. However, the RIPPLES survey also lacked questions related to a user’s dissatisfaction 
with the status quo. Since the target population had already adopted ePortfolios, the condition of dissatisfaction with 
the status quo was addressed through additional questions regarding rationale for adoption. 

Inconsistent response scales for the seven elements in the original RIPPLES survey were also of concern. 
Inconsistent scales can cause respondent confusion, demand longer response time, and may result in erroneous 
selections. Thus, all close-ended questions for the seven RIPPLES elements were rewritten as Likert scale 
statements with the options of strongly agree, agree, neutral, strongly disagree, disagree, and don’t know/unsure. 
These statements were followed with an open-ended question asking participants to comment on their choices, 
particularly those reflecting a position of “disagree” or “strongly disagree”. Figure 1 illustrates the revised RIPPLES 
survey for one of the seven elements, the resources element. 
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RESOURCES: The next set of statements relate to two resources (money and time) for adopting 
and using electronic portfolios. 


The resources (money and time) available for adopting electronic portfolios as a technology at 
your university are at an appropriate level. 

3 Strongly Agree 

3 Agree 

3 Neutral 

3 Disagree 

3 Strongly Disagree 

3 Don't Know/Unsure 

The resources (money and time) of your university related to electronic portfolios are allocated in 
an appropriate way. 

3 Strongly Agree 

3 Agree 

3 Neutral 

Disagree 

3 Strongly Disagree 
Dont know/Unsure 


The resources (money and time) of your university and the way those resources are allocated 
act as an enabler to the use of electronic portfolios. (An enabler makes an innovation easier to 
implement.) 

3 Strongly Agree 

3 Agree 

3 Neutral 

3 Somewhat Disagree 
3 Disagree 
Dont Know/Unsure 


Resources (money and time) are important to the successful use of a technology innovation, 
such as electronic portfolios. 

3 Strongly Agree 

3 Agree 

Neutral 

3 Somewhat Disagree 
3 Disagree 
Dont Know/Unsure 


Is there anything else you would like to tell me regarding resources (money and time) and their 
importance to electronic portfolio adoption and use? (For example, if you responded 'disagree* or 
'strongly disagree' to any of the statements in this set. you might use this space to explain your 
choice.) 


/ 

Figure 1. Resources section of revised RIPPLES survey. 
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Second, the demographics section of the original RIPPLES survey was revised to include only the four 
questions perceived as relevant to the ePortfolio study: gender, age, professional rank, and highest degree held. In 
addition, this section was relocated to the end of the survey, allowing participants to respond to the most 
intellectually demanding and study-relevant questions first and immediately after reading about the study’s purpose 
and details of informed consent. Figure 2 illustrates the revised demographics section. 

PART THREE: Demographic Information 

Please supply the following information regarding your demographics. 


Gender 
0 Male 
Female 


Age 

o 20-29 
o 30.39 
o 40-49 
0 50-59 
o 60-69 
70 or above 


Professional Rank 
0 Professor 
Associate Professor 
0 Lecturer 

° Adjunct Instructional Faculty (Part-time, Non-Tenure Track) 
0 Assistant Lecturer 
° Staff Associate 
Administrator 
0 Other 


Highest Degree Held 

< Doctorate 

< Masters 

< Bachelors 

< Other 

Figure 2. Demographics section of revised RIPPLES survey. 

Third, the opinion section of the original RIPPLES survey was modified to include a question that asked 
participants to rank each of the seven RIPPLES elements in order of importance to their ePortfolio implementation 
efforts. Adding this question enabled the researchers to analyze RIPPLES elements both within each RIPPLES 
element and across all seven elements. Figure 3 illustrates the revised opinion section. 
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OPINION: This next set of questions relate to your opinion about specific enablers and barriers to 
adopting and implementing electronic portfolios. 


In your opinion, what are the fwo biggest barriers that prevent users such as yourself from 
adopting and using electronic portfolios? (A barrier makes an innovation harder to implement.) 




In your opinion, what are the fwo biggest enablers that make it easier for users such as yourself 
to adopt and use electronic portfolios'’ (An enabler makes an innovation easier to implement.) 




If you were in charge of helping faculty at your university with adopting and using electronic 
portfolios, and you had unlimited resources, how would you do it? 


x: 


Please rank the following seven elements regarding the adoption and implementation of electronic 
portfolios by order of importance (FIRST place being of highest importance, etc.). 

(Note: You will need to use your mouse to drag and drop the elements in order to change their 
order.) 


Resources (defined as the time and money needed to adopt and use electronic portfolios) 


Infrastructure (defined as the overall technological backbone of an organization, including 
communication systems, networks, hardware, software, administrative and production 
facilities) 

People (defined as the human elements of your department or program, including goals, skills, 
talents, backgrounds, beliefs, opinions, and feelings) 


Policies (defined as the written and unwritten rules, practices, traditions, and regulations that 
govern your department or program s day-to-day operations) 

Learning (defined as the instructional outcomes of training experiences offered by Virginia 
Tech's ePortfolio Initiatives office) 

Evaluation (defined as the evaluation of important factors, such as learner achievement, 
impact of technology innovation, cost/benefit analysis, associated with a new program) 

Support (includes training, technical support, pedagogical support, and administrative 
leadership) 


Is there anything else you'd like to tell me about enablers or barriers to the adoption and use of 
electronic portfolios? 


A 


Figure 3. Opinion section of revised RIPPLES survey. 
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Recommendations for Future Survey Revisions 


The revised RIPPLES survey was administered and participant responses were analyzed as part of phase 
one of this Type 2 development research study, the analysis phase, by July 2013. Upon reflection, the researchers 
determined that further modifications could be made to the revised RIPPLES survey for future data collection efforts. 
The first recommendation is to remove either the “neutral” option or “don’t know/unsure” option from the Likert 
response scales. The inclusion of both options is not necessary and seems to have caused confusion for respondents. 
A second recommendation is to reduce the length of the survey by removing any questions that seem redundant or 
repetitive, particularly among the Likert scale questions for each RIPPLES element. The final recommendation is to 
modify the RIPPLES acronym or remove the RIPPLES acronym from participant view. Survey respondents seemed 
to find the RIPPLES acronym and its corresponding element terms and definitions confusing, stealing time and 
attention away from responding to actual survey questions themselves. 

Overall, the original RIPPLES survey model, grounded in implementation aspects of DOI theory, served as 
a very useful basis for creating a modified survey instrument for the ePortfolio implementation study. The 
researchers extend their gratitude to the originators of the RIPPLES model and hope that the survey modifications 
discussed in this paper will inform future applications of RIPPLES. 

Conclusion 

As education continues to embrace the use of technologies for teaching and learning, institutions of 
learning grapple to embrace new technologies without well-conceived integration plans in place. DOI theory has 
much to offer regarding how members of an institution adopt and implement such innovations into current practice. 
The RIPPLES model offers a useful framework for how important implementation elements of DOI theory can be 
operationalized. In revising the RIPPLES survey, the goal was to create an even more robust theoretical model and 
corresponding instrumentation for understanding instructional technology integration at one university. The 
realignment of the RIPPLES model with Ely’s eight conditions for implementation makes more explicit the DOI 
principles at play in ePortfolio implementation at one institution, ultimately assisting in defining a framework that 
can now be communicated to and used by other institutions to improve the instructional technology adoption and 
implementation process. 
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Motivation for the Study 

The rapid development internet-based technology along with the availability of user-friendly video capture and 
editing tools expanded the use of instructional video modules from fully online courses (e.g. Gosmire, Morrison, & 
Osdel, 2009), to blended courses (Sloan Consortium, 2007, Twigg, 2003), and to more recent enhanced live active 
learning strategies such as flipped or inversed classroom (Strayer, 2012; Zappe et ah, 2009). The research associated 
with instructional video modules ranged from production and usage evaluation (Copley, 2007; Griffin, Mitchell, & 
Thomson, 2009), to specific teaching strategies (e.g. Lawson, Bodle, & McDonough, 2007; Schrader, et ah, 2003; 
Shephard, 2003), or comparative use (e.g. Moreno & Ortegano-Layne, 2008). However, there is a lack of research 
focusing on how various stimuli included in an instructional video can promote and sustain students’ attention. 

Using the principles of cue-summation theory (Severin, 1967) we designed an experiment to study learners’ reaction 
to two types of cues introduced in video lectures. Cue-summation theory suggests that using redundant information 
increases the efficacy of the learning process (Brashears, 2004) as compared with the same information presented 
separately. The application of this theory in the field of human-computer interaction suggests multiple cues 
presented in both audio and video channels in online lectures improve information processing performance through 
faster and automatic associative processing (McNab, 2009) which, in turn, increases the amount of time learners can 
dedicate to understanding the concepts. 

For this study we designed a set of video lectures with embedded visual events, or cues (Hahn, 1973). Attention to 
these visual cues was evaluated by measuring changes in heart rate (Graham & Clifton, 1966). Orienting response to 
visual events in media have been shown to produce a decrease in heart rate (Reeves et al., 1999), with research 
showing this being an effective measure of short term attention when watching television programs (Lang, 
Newhagen, & Reeves, 1996). The main motivation of this study was to analyze the potential impact of two specific 
cues: instructor’s presence and gestures and respectively graphics (e.g. arrows) designed to capture students’ 
attention, two visual cues frequently used in the design of asynchronous video lectures. 

Instructional materials 

Instructional materials used in this study were part of a short professional development introductory course in civil 
engineering. The master instructional video was recorded using Chroma Key compositing technique (green screen); 
the video editing allowed superposing the instructor over the PowerPoint slides, with him actively pointing toward 
important parts of the information shown on the slide, which produced a viewing experience similar to physical 
classroom participation. 

From the master instructional video we selected three short 3 to 3.5 minutes videos sequences to create the 
experimental treatments: 1) introduction, 2) presentation of a basic concept, and 3) presentation of an applied 
concept. For all three videos. Power Point slides with voice-over narration were used as control, while the two 
treatments included the following specific visual cues: instructor presence and gestures or arrows appearing on the 
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screen (Fig. 1). The two specific cues complemented non-specific cuing elements such as the text and graphic on the 
lecture slides (visual cues) or the instructor’s narration (audio cues), experienced by all participants. The nine 
resulted short instructional videos were used in the development of the experimental research used in this study. 



(a) 


(b) 


Figure 1. Sample screenshots: (a) instructor presence and (b) cuing arrows 


Methods 

Research Design 

For this study we used a completely randomized design where subjects were assigned to one of the six distinct 
experimental designs generated by combining nine short instructional videos described in the previous section 
(Table 1). 


Table 1. The nine short videos, grouped in six distinct experimental treatments 



Video 1 
Introduction 

Video 2 

Basic Concepts 

Video 3 

Concept Application 

Treatment 1 

Instructor on 

Cuing arrows 

Voice-over slides 

Treatment 2 

Instructor on 

Voice-over slides 

Cuing arrows 

Treatment 3 

Cuing arrows 

Instructor on 

Voice-over slides 

Treatment 4 

Cuing arrows 

Voice-over slides 

Instructor on 

Treatment 5 

Voice-over slides 

Instructor on 

Cuing arrows 

Treatment 6 

Voice-over slides 

Cuing arrows 

Instructor on 


Participants 

To avoid any bias associated with participants’ familiarity with the instructor, 69 college students (32 males) form a 
different campus were recruited using a snowball sampling method. Participants were randomly assigned to one of 
the six treatments resulting in 10 to 12 participants for each treatment. 
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Participation was voluntary and a $10 meal ticket was provided as compensation. A screening questionnaire ensured 
that all participants were free of major motor, auditory or visual impairments and they were not currently 
undergoing any drug treatment for neurological or psychiatric disorders. 

Research Instrumentation and Procedures 

The experiment was conducted in a dedicated facility. Background noise level was minimal and ambient 
illumination and temperature was similar for all participants. The information was presented on a 19” CRT screen 
situated about 36” from the subject’s eyes at about eye level. The participants were seated, in a comfortable position 
(Figure 2). Philips noise cancelling earphones were used to convey the audio from the video. Participants were able 
to adjust the volume to a comfortable level. 




(a) (b) 

Figure 2. Research Laboratory Facility 
(a) Participant Setting; (b) Control Room 


A Pentium III PC running MediaLab and DirectRT (Empirisoft, 2007) was used to deliver both visual and auditory 
stimuli to the participants. ECGs were recorded using three 10mm Ag/AgCl electrodes located on both forearms 
(grounding electrode on left arm) and connected to a BIOPAC MP150WS data acquisition system. AcqKnowledge 
software (BIOPAC, 2007) running on a G4 Power PC Macintosh (Apple) was used to perform real-time ECG 
recording (at 10 ms interval). 

A training video (short animation sequence) presented at the beginning of the experiment was used to familiarize 
participants with the technology and the setting, reduce anxiety and help them and the researcher make adjustments 
to the equipment in order to provide optimum delivery and recording parameters. 

For each of the three categories of video modules (introduction, basic concept and concept application) the 
researchers selected as attention cues those times where the instructor either entered in the viewing area or he 
actively pointed toward a specific area on the slide. An interval of +/- 200 milliseconds around each selected cue 
was then considered for analysis. The visual cues were time-synchronized across all the experimental videos. 
Students were randomly assigned to one of the experimental treatments. At the end of the experiment we also asked 
participants which video they liked most and least and why. Their evaluations were used to triangulate our 
experimental findings. 


Data Analysis 

The orienting reflex (OR) was first described by Pavlov (1927) as a motor investigative response to an abrupt 
external stimulus. Subsequent studies linked the OR to cognition and its role in ensuring “contact with the stimulus” 
in order to facilitate central processing. Research has determined that OR is elicited by stimuli novelty, as compared 
to an established reference system. These two are of a dynamic nature, with the reference parameters constantly 
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adapting in response to external stimulation, thus generating habituation and, in time, subsequent decrease of OR to 
stimuli that fail to deliver enough novel information. 

Furthermore, research by Sokolov (1963) determined that OR is also influenced by the significance of the stimulus, 
not only by its novelty. That is, we tend to be more attentive to external stimuli that have a known significance to us 
than to those stimuli that are totally novel to us. These findings highlighted the role of OR for examining attention. 
Changes in heart rate (HR), and specifically HR deceleration immediately post stimulus administration, has been 
shown to correlate with the OR as a measure of attention. Van der Molen, Somsen and Jennings (1996) further 
postulated that observed OR as identified by HR deceleration is increased under voluntary attention and anticipation 
of an incoming event. 

The experimental treatments in the current study use video and audio sequences, which represent a multitude of non- 
specific stimuli. Considering the non-threatening nature of the study and the commonality of the situation for our 
subjects (attending a video lecture as a college student) the only specific cuing, elements (from a presentation, not 
from a knowledge perspective) introduced in the videos were abrupt arm movements by the instructor pointing to a 
specific area of interest on the slides or the abrupt apparition of colored highlighting arrows. The data analysis for 
this study was focused on observed changes in HR as a measure of OR around these specific moments, in an attempt 
to evaluate the value of either type of information presentation on being perceived as novel by the subject, thus 
generating an OR and promoting increased attention to the lecture. 

The data collection process produced a relatively large dataset, with more than two million recorded data points. 
Because such large datasets are relatively difficult to manipulate with traditional desktop statistical software 
application, for analysis we used a combination of database software applications (MySQL) and programming 
languages (SQL, Python, and R). 

Data cleaning and validation was performed using SQL and Python programming languages. Using the R 
programming language for statistical computations, new variables (z-scores, average z-scores, grouping variables) 
were calculated and added to the dataset. Finally, for each event a graphical representation was generated for visual 
analysis. Following an initial visual inspection of the generated graphics, for each event include in the analysis, a 
time interval was defined to capture its effects. These intervals were initially set to 300 milliseconds before and after 
the event and another set of graphics was generated. A more in-depth analysis of the instructional activity around the 
cueing points provided the basis for adjusting these intervals in an attempt to capture the entire range of effects. In 
situations when multiple overlapping effects were observed, additional subsets were defined to ensure a usable 
regression analysis. Finally, an updated graphical representation was generated and analyzed. 


Results and Discussions 

Researchers theorized that visual cues will have an impact on subjects’ attention as measured by the cardiac 
frequency. It is expected that sustained attention will be manifested through a longer orienting response following 
the target visual cues, depending on the type of cue offered (instructor’s gestures or cueing arrow) as compared with 
the control short videos (voice over slides). Orienting response is defined as the decrease in cardiac frequency 
following the visual cues. 

The remaining of this section will present visual analyses of the average cardiac frequency for major cuing points in 
the video lecture modules used in this study. Pre and post-cue average cardiac frequency will be analyzed to identify 
the length of the orienting response and the fastness of the recovery post-cure when the instructor was present on the 
screen, cueing arrows were showed or a combination of the two. 

Impact of Non-Specific Cuing Elements 

The inherent richness of video-based instructional materials continuously exposes learners to visual (text and 
graphics) and audio (instructor narration) cuing elements. Therefore the first event to analyze is a video sequence 
that included a new text section on the slide narrated by the instructor (Figure 3). With all subjects being exposed to 
the same non-specific cuing elements, this point can be considered a control event for our experiment. 
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The graphical representation of the mean z-scores of the cardiac frequencies for all three experimental groups 
clearly shows a similar structure of orienting responses following the non-specific cuing element (see Figure 3 c). 


Intersection problem basics 


• Form triangle that describes 
situation where two of the vertices 
are the two stations and the 
solution is the third vertex 


ntersection problem basics 


• Requires basic triangle solving 
skills: law of sines, law of cosines 
pljs other miscellaneous 
expressions 


(a) 


(b) 


Event 

new text on screen 



Figure 3. Orienting response for non-specific cuing (same for all treatments): new text section on the screen. 

(a) video screenshot before; (b) video screenshot after (c) orienting response (average cardiac frequency) 

Impact of Instructor Presence on Screen 

The second aspect we were interested in was if the presence of the instructor on the screen, when no actual 
presentation gesture are present, will produce a stronger orienting response (OR) than that produced by the non- 
specific cues in the video. We selected as the event instructor’s entry in the screen in the first part of the introductory 
video that had no significant text or graphic showing in the background (Figure 4 a, b). Only one of the three 
experimental groups were exposed to this event. The graphical representation of the average z-scores of cardiac 
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frequency, curve (1) in Figure 4c, clearly shows that the presence of the instructor created a stronger oriented 
response. The steeper regression line associated with the measurements for the treatment group (Figure 4 c) 
strengthens this finding. 



Event: Instructor in 



(C) 

Figure 4. Introductory video: the instructor standing as the only target cuing: (a) video screenshot before the event; 
(b) video screenshot after the event, viewed only by the instructor group; (c) orienting response (average 
cardiac frequency) for all three experimental groups 

The stronger orienting response shown in Figure 4c, indicates that, when the non-specific cues in the video are 
minimal, the presence of the instructor in the video has the potential to engage students in the instructional process. 

Impact of Specific Cues on a Text-Based Narrated Slide 
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The next sequence of events we analyzed was the inclusion of abrupt arm movement synchronized with cuing 
arrows on a text-based narrated slide as the control group, followed by slower arm movements across the same text 
section that was initially marked by the cuing arrow (Figure 5 a, b). 



(a) (b) 


£ 

8 

<0 

I 

<D 

CO 

a: 

-tr 

(0 

<p 

X 


Events: Events: 

instructor pointing hand moving 

arrow showing on screen arrow static on screen 



(1) instructor (2) arrows (3) voice-over slides 

(c) 

Figure 5. Orienting response with both the instructor and cuing arrows, (a) video screenshot with the instructor; (b) 
video screenshot with the associating cuing arrow; (c) orienting response (average cardiac frequency) 

The graphical representations of the average z-score of the cardiac frequency in Figure 5 c show that instructor’s 
initial abrupt ami movement produced a stronger short-term orienting response (see area “A” on Figure 5 c), but 
overall the three regression slopes were similar after this initial event, which shows that the cuing power of the non- 
specific cues was strong enough to attenuate the impact of specific cues. However, the presence of the instructor and 
of the cuing arrow produced a longer orienting response than the voice-over slides. That is, as shown in Figure 5 c, 
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the slope associated with the mean z-score of the cardiac frequency for the voice-over slides group (graph 3) 
changed toward the end into a positive slope (recovery), while the other two regression lines continued on a negative 
slope (longer OR). 

Comparative Impact of Instructor Gesturing and Dynamic Cuing Arrows 

The final event analyzed in this paper is similar to the previous one, but differs in that the non-specific cuing 
elements are stronger (diagrams as opposed to text) and the cuing arrows are switching positions following 
instructor’s major gestures (Figure 6). 



(c) 


Event 

change in all arrows 


(d) 



(1) instructor (2) arrows (3) voice-over 

slides 

(e) 
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Figure 6. Orienting response with both the instructor and dynamic cuing arrows, (a) the instructor before; (b) the 
instructor after; (c) cuing arrows before; (d) cuing arrwos after (e) orienting response (average cardiac frequency) 
The graphic representations of the mean z-scores of the cardiac frequency in Figure 5 e indicate that previous to the 
cueing arrow changes in position the orienting response and the associated recovery followed a similar pattern (see 
area “A” in Figure 5 e). However, the dynamic change of the cuing arrows produced a significant additional 
orienting response (see area “B” in Figure 5 e) that was not present when the specific cue was the continuous 
gesturing of instructor’s arm. 

Conclusion 

A visual analysis of the average cardiac frequency (mean z-scores) for each treatment, pre and post cue, showed 
that, for the majority of the studied cues, the longest orienting response was generated by those videos where the 
instructor was present on screen, followed by the presence of cueing arrows. Voice over only videos showed, in 
general, the shortest duration of the observed orienting response and the fastest recovery post-cue However, the 
presence of dynamic text chunks or of diagrams on the background slides can also produce strong orienting 
responses in a simple voice-over narration. 

Therefore, the visual analysis indicate that, based on the observed orienting response, the presence of either the 
instructor or the targeted cuing arrow in an online instructional video has the potential to increase attention to the 
presented material. This was also confirmed by the participants’ subjective evaluation of the videos. Most 
participants preferred the short video in which the instructor was present, followed by the short videos with cuing 
arrows. Future work will focus on grounding these findings in statistical analyses appropriated for the nature of the 
measurements and the richness of the existing dataset. 
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Abstract 

This completed study examines how learners’ emotions (valence and arousal), induced by video clips, 
influence learning performance and mental effort in an animated instruction with different modalities (written-text 
vs. spoken-text). In each modality condition, participants were randomly assigned to four groups (a) calm positive, 
(b) calm negative, (c) arousing positive, and (d) arousing negative emotions. The findings revealed that recall test 
scores were significantly greater with arousing than calm emotion only in the written-text condition while emotion 
did not influence mental effort. 

Introduction 

Individuals can have emotional feelings before or during learning activities (Desmet, 2002). Emotion 
consisting of two dimensions can vary from positive to negative (valence) and from calm and arousing (arousal) 
simultaneously (Russell, 2003). Thus, individuals can feel a mixed feeling, such as calm positive emotion or 
arousing negative emotion. Some studies suggested the superiority of positive emotion on motivation to learn and 
learning outcomes (e.g., Goetz, Pekrun, Hall, & Haag, 2006; Pekmn, Goetz, Tiltz, & Perry, 2002). Other studies 
suggested that arousing emotion can enhance learning (e.g., memory) (e.g., Falk & Gillespie, 2009). However, there 
is little consideration on how the coexisting, two dimensions of emotion interactively influence learning 
performance. Thus, this study aims to examine how valence and arousal interactively influence learning 
performance and the efficiency of mental effort. 
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Literature Review 


Motivated Cognition 

Human emotion, motivation, and cognition (including learning) are intertwined and influence each other. 
According to a motivated cognition perspective (Lang, 2006), human has two basic motivational systems — 
appetitive and aversive systems. The both systems automatically activate in response to motivational relevance of 
information. The appetitive and aversive activation occurs by pleasant (e.g., opportunity) and unpleasant (e.g., 
threat) information, respectively, and elicit positive and negative emotions. 

Each purpose of appetitive and aversive activation is to remember opportunity and threat information. In a 
safe environment, human has innate desire to explore pleasantness. Thus, as shown in Figure 1 (calm levels of 
arousal), the appetitive system greater activates than the aversive system. However, with an increase in threat 
(arousing levels), the degree of activation is greater for aversive than appetitive activation to protect individuals 
themselves. 

Overall, arousing emotion results in greater activation of both appetitive and aversive systems. Thus, 
greater activation means to better memory of the information. However, when emotional states are highly arousing 
negative, the aversive system turns down to transfer the information into memory. In learning environments, it 
would not be recommended that learners face life-threatening information. Thus, the current study examines a low- 
to-moderate range of arousal by operationally defining calm and arousal levels. 

Emotion in Learning Environments 

Several studies provided evidence that emotion affects learning. Emotion can be induced by external mood 
induction (e.g., emotional video clips) (Um, Plass, Hayward, & Homer, 2011; Wolfson & Case, 2000). Isen and 
Reeve (2005) found that learners’ positive emotion, induced by a three-dimensional puzzle play, enhanced 
motivation to learn and perceived enjoyment. Falk and Gillespie (2009) found that arousing emotion by direct 
experience at a science center resulted in better memory and comprehension for learning content. However, these 
studies underestimated the effects of negative emotion on learning. 

Instructional animation is beneficial to visualize complicated processes (e.g., how a car’s braking systems 
work, how pumps work) and dynamic changes of a movement (e.g., how lightning develops) (Mayer & Moreno, 
2003). The animation can be displayed with explanatory on-screen text or narration. Studies on multimedia 
principles have suggested that animation with spoken-text is better effective in recall of content than the animation 
with written-text because of split attention (Mayer, 2009). However, there is little research on how learners’ emotion 
affects their learning in different modality conditions. Therefore, this study has the following questions: 

• RQ1 : Do valence and arousal differently influence learning performance and mental effort for test between 
written-text and spoken-text animations? 

• RQ2: Does learning performance differ in valence and arousal levels in accordance with motivated 
cognition (see Figure 1)? 

• RQ3 : Does mental effort for test differ in valence and arousal levels in accordance with motivated 
cognition (see Figure 1)? 



Figure 1. Motivated Cognition 
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Method 


Data Collection 

Participants in this study were 206 undergraduate students (N= 106 for a written-text version; IV = 100 for 
a spoken-text version) from a large southwestern university. This study used three between-subjects independent 
variables: (a) Modality: written/spoken, (b) Valence: positive/negative, and (c) Arousal: calm/arousing. The written- 
text condition presented four explanatory written sentences on the screen in a sequential order while showing an 
appropriate animation. The spoken-text condition had only narration without presenting the texts during the 
animation. As an intervention, a preceding video clip with four different emotional tones in valence and arousal 
appeared before the learning content: (a) calm positive: nature view, (b) calm negative: slum street, (c) arousing 
positive: sports events, and (d) arousing negative: nature disaster. 

Dependent variables were (a) a recall test for learning content and (b) mental effort during the test. The 
recall test asked the participants to write down an explanation of how lightning works. Each sentence contained two 
meanings and therefore could be scored from 0 to 2 points. Since there were four sentences, a participant could 
receive from 0 to 8 points in the recall test. Mental effort was measured by a one-item 9-point scale (1 = extremely 
low; 9 = extremely high) (i.e., Please indicate how much mental effort you invested in this test). 

This study followed four phases below: 

• emotional video clip (treatment) 

• self-ratings of valence and arousal (for manipulation check) 

• learning content 

• assessment (questions about recall test and mental effort) 

Manipulation Check 

Participants were asked to indicate how positive, negative, and exciting the video clip they just watched on 
a three 9-point Likert scale questionnaire (e.g., 1 = extremely not positive; 9 = extremely positive). Three t-test 
analyses were conducted. Positive valance was significantly greater for positive video clips (N = 104; M= 6.67, SD 
= 1.771) than negative video clips (N= 102; M= 2.25, SD = 1.426), 1(204) = 19.102, p< .001. Negative valence was 
significantly greater for negative video clips (N= 102, M= 6.68, SD = 1.941) than positive video clips (N= 104; M 
= 2.38, SD = 1.578), 1(194.32) = -17 .437, p < .001. Arousal was significantly greater for arousing video clips (N = 
103; M= 4.83, SD = 2.549) than calm video clips (N= 103; Mm 3.44, SD = 2.159). Thus, external video clips used 
in this study were successfully manipulated. 


Results 

To test research question #1, 2(Modality) x 2(Valence) x 2(Arousal) ANOVA analyses were performed on 
recall test scores and mental effort scores, respectively. 

Regarding recall test scores, significant main effects were not found in terms of modality (F(l, 198) = .514, 
p = .474) and valence (F(l, 198) = 3.769, p = .054). However, arousal had a main effect (i^l, 198) = 5.329, p = 
.022). Significant interaction effects were not found for modality x valence (F(l, 198) = 306, p = .581) and 
modality x valence x arousal (F(\ , 198) = .474. p = .485. However, a significant interaction effect was found for 
modality x arousal (F(\, 198) = 5.713, p = .018). 

Regarding mental effort scores, modality had a significant main effect (TjT, 198) = 4.017,/? = .046 < .05). 
However, no significance was found for valence (TjT, 198) = 1.383,/? = .241), arousal (TjT, 198) = 1.195,/? = .276), 
modality x valence (TjT, 198) = .009, p = .924), modality x arousal (F(l, 198) = .704,/? = .403), and modality x 
valence x arousal (TjT, 198) = .045,/? = .833). 

To test following research questions #2 and #3, we employed separate 2(Valence) x 2(Arousal) ANOVA 
and independent t-test analyses in each modality condition. Table 1 describes the means and standard deviations for 
all measures. 
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Table 1. Means and Standard Deviations for All Measures 

Modality 



Written-Text 
(N= 106) 

Spoken-Text 
(N= 100) 




Arousal 

Arousal 




Calm 

Arousing 

Calm 

Arousing 

Recall Test 

Valence 

Positive 

4.04(1.16) 

5.19(1.78) 

4.48(2.18) 

4.12(1.56) 

Sores 

Negative 

3.42 (1.21) 

4.58 (1.72) 

3.80 (2.24) 

4.12 (1.94) 


Mental Effort 

Valence 

Positive 

6.22(1.40) 

6.52 (1.72) 

5.92 (1.53) 

5.92 (2.20) 

Sores 

Negative 

6.42 (1.42) 

6.42(1.42) 

5.56 (1.76) 

5.68 (1.91) 



* Standard deviations are presented in parenthesis 


On recall test scores in the written-text modality condition, significant main effects were found for valence 
(F(l, 102) = 4.431 , p = .038) and arousal (FjT, 102) = 15.791, < .001). However, there was no interaction effect 
between valence and arousal (^(l, 102) = .000,/? =.992). Specifically, independent t-test analyses found that 
arousing positive emotion was significantly greater than calm positive emotion (7(44.749) = -2.812 ,/? = .007). 
Arousing negative emotion was significantly greater than calm negative emotion (t(50) = -2.796 ,p = .007). Yet, 
there were no significant differences between calm positive and calm negative emotion (1(51) = 1.890,/? = .064) and 
between arousing positive and arousing negative emotion (1(51) = 1.264,/? = .212). On mental effort scores, 
significant effects were not found for valence (FjT, 102) = .751,/? = .388), arousal (T^l, 102) = .124,/? = .124), and 
valence x arousal (7^(1, 102) = .294,/? = .589). 

On the other hand, on recall test scores in the spoken-text modality condition, no significant effects were 
found for valence (F (\ , 96) = .723 , p = .397), arousal (FjT, 96) = .003,/? = .960), and valence x arousal (^(l, 96) = 
.723 , p = .397). On mental effort scores, no significant effects were also found for valence (F(l, 96) = .648,/? = 

.423), arousal (F(\, 96) = .026,/? = .872), and valence x arousal (F( \ , 96) = .026,/? = .872). 

Discussion and Conclusions 

The results revealed that valence and arousal had significant effects on recall test scores in the written-text 
condition whereas no effect was found in the spoken-text condition. Human voice has unique emotional tone. Thus, 
we conjecture that the narration in the spoken-text instructional animation might dominate the learner’s emotional 
state (Piwek, Petrini, & Pollick, 2012) and attenuate the emotion induced by external video clips. Moreover, 
auditory processing needs a substantial amount of cognitive load because the learners have to endeavor to 
understand what they just heard but not exists as text (Hampson & Morris, 1996). Therefore, the limited pool of 
working memory might be occupied by the auditory processing rather than pre-existing emotional processing. 

In line with motivated cognition studies (Lang, 2006), the positive effects of arousal emotion were found in 
both positive and negative feelings. Regarding valence, recall test scores partially followed the motivated cognition 
pattern (See Figure 1). For example, in calm levels, positive and negative emotion yielded similar scores in recall 
test. In arousing levels, however, the positive emotion group outperformed the negative emotion group. It seems that 
arousing positive emotion lasted longer than arousing negative emotion. We assume that the sports game 
commentator’s description of the exciting events made the participants keep their emotion longer while the other 
video clips did not contain any spoken description. Meanwhile, valence and arousal effects did not influence mental 
effort during the test. Therefore, it can be said that arousing emotion is an efficient method to increase learning 
outcomes since it produces higher recall scores with the similar cognitive resources. 

In computer-based learning environments, instructional designers have not considered learners’ emotional 
states, but our findings imply that for better learning outcomes, they could use a way to increase learners’ arousal 
levels by presenting such emotional materials before learning content (i.e., the context of using written-text 
animated instruction). This study revealed several limitations in the manipulation on arousal levels. We examined a 
narrow range of arousal levels. Future studies can explore multiple degrees of arousal to find interaction effects 
between valence and arousal. Additionally, we used external video clips with emotional tone. Future studies may use 
pictures, music, or other emotional materials. More detailed implications and limitations will be presented at the 
conference. 
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Abstract 

The 21 st century has witnessed exponential growth in the availability of information and communication 
technology accompanied by a proliferation of new and exciting ways of using the internet and the world-wide web. 
This has not been lost on the Caribbean, as more and more educational institutions are taking or seeking to take 
advantage of the possibilities provided by the internet and the web for reaching and teaching students. 

The aim of this research was to determine students’ readiness for operating in the online learning 
environment, as well as to ascertain their attitudes to online learning, in general, and their perceptions of the 
usefulness of online learning to Academic Writing, in particular. This was done in order to inform the process of 
integrating online learning modalities in the delivery of Academic Writing modules at a university in Jamaica. A 
quota of 30 undergraduates representing each of the seven faculties/colleges made up the sample, which totaled 210 
students. A survey design was employed with seven research questions guiding the study. The data collection 
instalment was an Online Learning Questionnaire with section(A) consisting of questions designed to capture 
demographic, institutional, experiential and socio-economic data, and section(B) consisting of four point Likert- 
type scale ranging from ‘Strongly Disagree’ to ‘Strongly Agree’. 

Results indicated that there was a high level of readiness among 63% of the students and 48% had a poor 
attitude towards online learning. The findings also revealed that (1) the easier it is to procure funds for education, the 
better a student’s level of readiness for online learning (2) as the students grow older, their readiness improves (3) 
part-time students have a better attitude to online learning than their full-time counterparts (4) the most preferred 
learning modality was face-to-face. 


Introduction 

Online learning in Jamaica and the wider Caribbean 

The 21 st century has witnessed exponential growth in the availability of information and communication 
technology accompanied by a proliferation of new and exciting ways of using the internet and the world-wide web. 
This has not been lost on the Caribbean, as more and more educational institutions are taking or seeking to take 
advantage of the possibilities provided by the internet and the web for reaching and teaching students. 

The thrust to reach students (both literally and figuratively) and teach them via online instruction in 
Jamaica is evidenced by the e-learning Jamaica Project targeted at secondary level students, and the use of online 
modalities ranging from web-assisted to distance/fully online at the major universities and colleges. (ODL, 2008; 
UWI Open Campus, 2012; Woodall,2012; CUPIDE, 2012). The Caribbean Knowledge and Learning Network, 
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CKLN, an inter-governmental Agency of the Caribbean Community, CARICOM, which was established to assist 
Caribbean member states to enhance their global competitiveness by upgrading and diversifying the skills and 
knowledge of human resources through greater regional collaboration and connectivity, has been actively engaged in 
the drive to establish national research and education networks throughout the region. (CKLN, 2013). In its 2012 
report, the Caribbean Universities Project for Integrated Distance Education (CUPIDE), commissioned by 
UNESCO, revealed that, “The CKLN scheduled strategic planning in distance education for Haiti in the first quarter 
of 2008 while the University of Guyana, UG, contracted the UWI’s, Mona Information Technology staff to train the 
UG technical staff.” (p. 83). Thus, from the Bahamas in the north to Guyana and Suriname on the South American 
continent, the member states of CARICOM have been collaborating in active pursuit of the full integration of e- 
learning. Before the advent of the CKLN, however, the regional university, UWI, had been engaged in distance and 
blended learning via the University of the West Indies Distance Teaching Enterprise, UWIDITE, which has evolved 
into the present UWI Open Campus. Jamaica’s national public university, the University of Technology, Jamaica, 
has also been offering a Bachelor of Education programme via distance learning to students in St. Vincent and the 
Grenadines. 

Online learning in all its modalities, from web-assisted to distance/fully online, is alive in the Caribbean; 
however, research on online education in the Caribbean, specifically in the areas of student readiness and attitudes, 
appears not to have kept pace with the thrust for utilizing online modalities in education. Indeed, at the time of 
writing, this researcher was able to find only three published studies on attitudes to online learning in the Caribbean 
and none on student readiness, after several weeks of search. 

Online learning and the university context 

The university under study is increasing its efforts to advance online learning, especially with regard to 
encouraging and supporting all its faculties and colleges in the utilization of the full gamut of online learning 
modalities. This university’s Office of Continuing Education, Open and Distance Learning defines online learning 
modalities thus: 

All modules that are primarily (that is, with >70% of module hours) mediated by the web at a 
distance, supported by the selected Learning Management System (LMS) and have the majority of 
content, interactions and assessments conducted online, are defined as Distance Education 
Modules. Modules which extensively use the selected LMS (50-70% of sessions), supported by 
face-to-face sessions, and which integrate online assessment with face-to face sessions, are 
identified as Hybrid Online Modules. Modules which integrate the use of the selected LMS (30- 
50% of sessions) with face-to-face sessions, promote interaction online and make the selection of 
assessment modalities reflect the actual balance between online and face-to-face sessions, are 
identified as Blended Modules. Modules which promote use of online tools and online interaction 
with the Instructor, explore the use of the LMS (20-30%% of sessions) to supplement face-to-face 
sessions and may include online components as part of formal assessment, are identified as Web 
Enhanced Modules. Modules which use the selected LMS (10-20% of sessions) primarily to 
distribute content and to supplement face-to-face sessions, and do not recognise online 
components in assessment, are identified as Web Assisted Modules. (University of Technology, 
Jamaica, 2009) 

The School of Humanities and Social Science of the Faulty of Education & Liberal Studies currently 
delivers Academic Writing 1&2 modules via the traditional face-to-face, as well as the web-assisted learning 
modalities. The School is, however, increasing its efforts to also utilize the blended and distance online modalities 
in the delivery of Academic Writing modules by 2014. While this thrust is commendable, the instructional design 
ought to reflect input from major stakeholders, chief of whom are the students. As Beard, et al (2008) pointed out, 
decisions are often made about online teaching without the input of the students concerned, but studies have 
revealed that it is important to seek student input as doing this can provide insight that might otherwise have been 
lost on the designers of the courses. In this case, student input may take the form of suggestions and feedback 
regarding their experiences with the existing use of the web-assisted modality in the Academic Writing modules. 
Importantly, any curriculum design needs to consider the important variables of students’ attitudes and perceptions. 
A survey done by Bernard (2009) in which learners were asked to give their views about online teaching, unearthed 
various concerns, desires and expectations that were largely overlooked by the designers of an online course. As was 
expected, most of the respondents gave favourable ratings to the blended approach but statements such as, “Online 
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education is exciting only to students already steeped in a technology-based world”(p. 7) reveal some of the fears 
that those prospective students had. 

Rationale, Aim and Purpose 

Academic Writing is a general education module that students in all the seven faculties/colleges of the 
university have to complete as partial fulfillment of the requirements for the award of the Bachelors Degree. These 
students possess different skills, experiences, interests and motivation; therefore, serious consideration must be 
given to their readiness for, perceptions of and attitudes towards any learning experience that is relatively new to 
them. The aim of this research, therefore, was to determine students’ readiness for operating in the online learning 
environment in terms of their ICT skills, access, self-discipline and self-efficacy, as well as to ascertain their 
attitudes to online learning, in general, and their perceptions of the usefulness of online learning to Academic 
Writing, in particular. This was done in order to inform the process of integrating online learning modalities in the 
delivery of Academic Writing modules at the university under study. 

Research Questions 

1(a) What is the level of readiness among students for operating in the online learning environment? 

1 (b) Are there any significant relationships between gender, age group, faculty, enrollment status, ease 
of funding education and students’ readiness for operating in the online learning environment? 

2(a) What are students’ attitudes towards online learning? 

2(b) Are there any significant relationships between gender, age group, faculty, enrollment status, ease 
of funding education, satisfaction with online learning experience and students’ attitudes to online 
learning? 

3(a) What are students’ perceptions of the usefulness of online learning to the Academic Writing 
module? 

3(b) Are there any significant relationships between gender, age group, faculty, enrollment status, ease 
of funding education, satisfaction with online learning experience and students’ perceptions of the 
usefulness of online learning to Academic Writing? 

4 What are the students’ preferences among the learning modalities - face-to- face, web-assisted, 

blended and distance? 


Theoretical Framework 
The role of learning theories in online education 

As Knowlton (2007) opines, “It is a daunting task to theoretically frame the pedagogy of the online 
classroom.” (p.l). Citing (Draves, 1999), he further reveals that, “Synthesizing varied new and largely untested 
practices with equally varied educational theories could fill many volumes.” (Ibid). A review of relevant learning 
theory is, however, essential to any discussion regarding pedagogical design since learning theories form the basis of 
the learning models and approaches taken to achieve desired learning outcomes. As Anderson (2004) notes, “We 
need theories of online learning that help us to invest our time and limited resources most effectively.” (p. 34). Ally 
(2004) also reveals that studies have shown that there are significant learning benefits gained from e-learning but 
those same studies suggest that the reason for those benefits are not technology per se but the instructional strategies 
built into the learning experience. 

The three most popular theories of learning are Behaviourism, Cognitive and Constructivism; however, 
studies investigating learning theories and the technology driven classroom, for example, ( Knowlton, 2007; Martin- 
Stanley & Martin-Stanley, 2007; Mayes & de Freitas, 2006), have advanced the Constructivist learning theory as 
being the most effective. According to Martin-Stanley, et al (2007), “taken together, constructivist practice and 
technology offer compelling evidence of the benefits of educational innovation on student learning outcomes.” (p. 

3). At the heart of Constructivism is the student-centered approach, which, Knowlton argues, is a necessity if the 
goal of the online classroom is student learning. The student-centered approach is demonstrated in constructivist 
learning experiences, which provide students with the opportunity “to take personal responsibility, exercise 
initiative, and be in control in the instructional setting through a variety of learning experiences” (Snyder, Bolin, & 
Zumwalt, 1992, p. 415), a process very much in tandem with online learning. 
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The chief tenet of Constructivism is that understanding is gained through an active process of creating 
hypotheses and building new forms of understanding through activity (Mayes & de Freitas, 2006). Constructivist 
principles for designing teaching and learning activities include: 

• Ownership of the task 

• Coaching and modelling of thinking skills 

• Scaffolding 

• Guided discovery 

• Opportunity for reflection. (Mayes, et al, 2006, p. 16) 

The concept of scaffolding refers to support that is given to learners who are in the zone of proximal 
development. Citing Vygotsky (1978), Mayes, et al (2006) define the zone of proximal development as, “the 
distance between a learner’s current conceptual development (as measured by independent problem solving) and 
that learner’s potential capability, as measured by what can be accomplished under guidance or in collaboration with 
more capable peers. ’’(Ibid). (Peal and Wilson (2001) as cited in Mayes, et al, (2006), demonstrate zones of proximal 
development in the following features of the design of web-based learning environments : 

1 . Learning activities that are part of real or simulated activity systems, with close attention to the tools and 
interactions characteristic of actual situations. 

2. Structured interaction among participants. 

3. Guidance by an expert. 

4. The locus of control passes to the increasingly competent learners, (p. 19) 

In spite of the obvious consensus regarding the suitable fit of constructivism and computer-mediated 
learning, the Social Cognitive theory must not be thrown out as it helps to explain the cognitive processes involved 
in learners’ perceptions, which is central to the Technology Acceptance Model (TAM). In his Transformation theory 
on how people learn, Mezirow (1991) provides a nexus between the constructivist and cognitive schools of thought. 
According to Mezirow (1991), “learning is the process of using a prior interpretation to construe a new or revised 
interpretation of the meaning of one's experience in order to guide future action.” (p.12). This is central to the 
constructivist thought. He further explains that transformative learning entails “reflectively transforming the beliefs, 
attitudes, opinions, and emotional reactions that constitute our meaning schemes or transforming our meaning 
perspectives” (p. 223) This is a feature of the Social Cognitive theory. 

Perceptions, attitudes and the Technology Acceptance Model 

The Theory of Reasoned Action(TRA) assumes that behaviour can be controlled by an individual’s will 
and that perceptions and attitudes influence behaviour. (Chen & Huang, 2012; p.90). Drawing on the Theory of 
Reasoned Action, Davis (1986 ) developed the Technology Acceptance Model (TAM) , which postulates that the use 
of an information system is determined by behavioral intention and that behavioral intention is determined by the 
person’s attitude towards the use of the system, as well as by his perception of its utility. (Dillon & Morris, 1996). 
The TAM claims to predict the acceptability of an information system, which is determined by the perceived 
usefulness and the perceived ease of use of that system. As reported by Dillon, et al (1996), “research presented by 
Davis (1989) to validate his model, demonstrates that the link between the intention to use an information system 
and perceived usefulness is stronger than perceived ease of use.” (p.3) 

The Technology acceptance Model has been instrumental in online learning research and design; however, 
in a critical review of the TAM, Legris, et al (2003) concluded that, “ the TAM is a useful model but has to be 
integrated into a broader one which would include variables related to both human and social change processes, and 
to the adoption of the innovation model. ”(p. 1) 


Methodology 


Research Design 

A survey design with quantitative features was employed in this study. 

Participants 

The sample consisted of a quota of 30 students representing each of the seven faculties/colleges. Two 
hundred and thirty-five questionnaires were distributed among undergraduates completing Academic Writing in day, 
evening and weekend classes. Two hundred and twenty-four questionnaires were returned with 214 found usable. A 
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total of 2 10 were selected. The rationale for using a quota sample is that it was possible that very few students from 
the smallest faculties would have been captured in a simple random sample as a result of the over-representation of 
students from two large colleges. This would have rendered faculty as a variable almost meaningless. 

The sample showed a gender distribution of 58% female and 42% male. This is a reflection of the ratio of males to 
females in the university. 

Instruments 

The data collection instrument was an Online Learning Questionnaire with section(A) consisting of 
questions designed to capture demographic, institutional, experiential and socio-economic data and section(B) 
consisting of four point Likert-type scale ranging from ‘Strongly Disagree’ to ‘Strongly Agree’. Some items 
measuring students’ readiness on the Likert-type scale were adapted from Dray, et al (2011) Instrument to Assess 
Online Learning Readiness. This instrument went through a validation process, which, according to Dray, et al 
(2011), “resulted in not only a more rigorous instrument but one that more clearly defines ready and situates it 
within the literature on learner characteristics, digital divide, and information and communications technology (ICT) 
engagement.” 

The Cronbach alpha reliability coefficient for the Likert-type items on the Online Learning Questionnaire was .798. 
Data Analysis 

Data analysis was done using descriptive statistics, Pearson chi-square, Spearman’s rho and linear 
regression where appropriate. The research questions and findings are displayed below. 


Findings 


Online Learning Readiness 

Research Question 1(a): What is the level of readiness among students for operating in the online learning 
environment? 

Descriptive statistics were computed and the findings, as indicated in Figure 1, reveal that 63% had a high level of 
readiness for operating in the online learning environment. 



Figure 1: Showing level of student readiness for online learning environment 


Research Question 1(b): Are there any significant relationships between gender, age group, faculty, 
enrollment status, ease of funding education and students’ readiness for operating in the online learning 
environment? 


44 


Spearman’s rho correlation coefficients were computed. The correlation between ease of funding education and 
students’ readiness for operating in the online environment, as well the correlation between age group and students’ 
readiness for operating in the online environment were found to be significant. 

Correlation between ease of funding education and students’ readiness for operating in the online learning 
environment. 

The correlation revealed that there is a weak inverse significant relationship between ease of funding 
education and student readiness for online learning (.217, p<.05). This indicates that approximately 5% of the 
variation in student readiness for online learning can be explained by ease of procurement of funds for education. 
The inverse nature reveals that the easier it is to procure funds for education, the better a student’s level of readiness 
for online learning. 

Correlation between age group and students’ readiness for operating in the online learning environment 

There is a significant relationship between age group and readiness (.147, p<.05). The relationship is very 
weak and only 2% of the change in readiness can be attributed to a student’s age group. This means that as the 
students grow older, their readiness improves. 


Students’ Attitude to Online Learning 

Research Question 2(a): What is the attitude among students towards online learning? 

Descriptive statistics were computed and the results reveal that 48% of students had a poor attitude towards online 
learning. This represents almost a half of the respondents. Figure 2, displays the findings. 

Students' Attitude to Online Learning 

■ Low 
■High 

■ Medium 


Figure 2: Showing Students’ Attitude to Online Learning 



Research Question 2(b): Are there any significant relationships between gender, age group, faculty, 
enrollment status, ease of funding education, satisfaction with online learning experience and students’ attitudes to 
online learning? 

Relationship between Enrollment Status and Attitude to Online Learning 

There is a significant relationship between enrollment status and attitude to online learning. As outlined in 
Table 1, approximately 57% of full time students compared to 26% of their part time counterparts had poor attitude. 
However, a little over a half of part time students compared to full time (27.5) had mixed attitudes. Approximately 
16% of full time and 20% part time students reported a good attitude. This relationship, however, is weak and 
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approximately 7% of the variation in online attitude can be explained by enrollment status. In other words, this 
finding reveals that part-time students have a better attitude to online learning than their full-time counterparts. 


Table 1: Relationship between Enrollment Status and Attitude to Online Learning 


Attitude to Online Learning 

Enrollment Status 

Full time % (n=153) 

Part Time % (n=50) 

Poor 

56.9 

26.0 

Mixed 

27.5 

54.0 

Good 

15.7 

20.0 


(X 2 =15.520 (2) p<.05) 


Further analysis of a multiple regression showed a significant relationship which is moderate in strength 
between online attitude and the independent predictors of faculty, enrollment status, satisfaction with online learning 
experience and age group( F=2.050 (11) p<.05, R=.380, R Square =.144). Approximately 14% of the variation seen 
in online attitude can be attributed to the model of predictors. The factors which made this significant contribution 
are faculty and enrollment status, with the former having the greater influence. The factor ‘faculty’ had several 
categories of which FOBE was the only category which was significantly different(better attitude), in online attitude 
when compared to law, the reference group. For enrollment status , full time students have a worse attitude than 
their part time counterparts. 


Table 2: Factors Predicting Students Online Attitude 


Factor 

B 

Beta 

Sig. 

Faculty: 

FOSS 

.801 

.186 

.061 

FOBE 

.695 

.249 

.038 

COHS 

.244 

.087 

.461 

COBAM 

.596 

.242 

.055 

FELS 

.259 

.107 

.399 

FENC 

.028 

.010 

.932 

Enrollment Status: 
Full time 

-.517 

-.213 

.029 

Satisfaction: 
Not satisfied 

.227 

.101 

.215 

Age Group: 
20 and under 

-.154 

-.078 

.673 

21-25 

.141 

.069 

.697 

26-30 

.164 

-051 

.680 

Constant 

1.401 


.000 


F=2.050 (11) p=.028) R=.380, R Square =.144, n=146 


Reference category for faculty is law 

Reference category for enrollment status is part time 

Reference category for satisfaction with online learning experience is satisfied 
Reference category for age group is 31 and above 


Research Question 3(a): What are students’ perceptions of the usefulness of online learning to the Academic 
Writing modules? 
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Perceptions of Online Learning Usefulness to 
Academic Writing 


■ Bad 

■ Moderate 

■ Good 


Figure 3: Perceptions of Online Learning Usefulness to Academic Writing 

As displayed by Figure 3, 84% of the students had moderate perceptions of the usefulness of online 
learning to Academic Writing. This indicates mixed perceptions. In other words, most of the students see online 
learning as being useful to Academic Writing in some areas and not useful in others. 

Research Question 4: What are the students’ preferences among the learning modalities - face-to- face, 
web-assisted, blended and distance? 

As displayed in Figure 4, the most preferred learning modality is face to face with blended learning being a 
distant second in the order of preferences. Web-assisted follows blended learning closely; however, distant learning 
is in a distant last place as the least preferred of the learning modalities. 




Discussion 

Student readiness plays a critical role in any online learning endeavour, (Horng-Ji Lai ,2011; Song, L. & 
Hill, J. R.) as learning outcomes are highly dependent on readiness factors such as ICT skills, access, self-discipline 
and self-efficacy. (Hiemstra, 2006; Dray, 2011). It was with this in mind that student readiness for online learning 
was investigated in this study. It was expected that approximately 60% of the respondents would have been found to 
possess high readiness for online learning based on anecdotal evidence (factors such as access to ICT, self-direction. 
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self-efficacy and ICT skills considered). The findings have, however, revealed a slightly better than the expected 
percentage of students who are highly ready for online learning, which is encouraging. 

The literature abounds with research on the influence of demographic variables such a s gender and age on 
students’ online readiness, attitudes and perceptions, with many revealing that younger people and males, in 
particular, are perceived as the most technologically savvy and attuned (Bennett, S., Maton, K. & Kervin, L. 2008; 
Raturi, S., Hogan, R. & Thaman, K. H. 2011). In light of this, the expectation was that there would have been 
significant relationships between the demographic variables age and gender and readiness for online learning. The 
correlation with regard to age was found to be significant, with the findings indicating that the older age groups have 
a higher level of readiness than the youngest. This seems plausible since readiness for online learning involves 
characteristics such as self-direction, which is expected to be more developed the older one gets. 

With respect to the significant correlation between ease of Rinding and online learning readiness, it was 
discovered that those who reported that it was relatively easy for them to fund their university education had the 
highest level of readiness for operating in the online learning environment. Again, this is plausible as students who 
are struggling with finding funds for tuition and other expenses attendant with pursuing tertiary education are less 
likely to own computers or even have easy access to the internet. Also, more financially able students are likely to 
have been exposed to better preparation for operating in the online environment as a result of their ability to attend 
private schools or afford personal tutors. In a study conducted by Warschauer, Knobel & Stone (2004) in California 
high schools, it was revealed that, “Although student-computer ratios in the schools were similar, the social contexts 
of computer use differed, with low-SES schools affected by uneven human support networks, and irregular home 
access to computers by students.” (p.l) 

The significant correlation between enrollment status and attitude reveals that part-time students have a 
better attitude to online learning than their full time counterparts. Again, this was expected as learners who have full 
time jobs are more likely to view favourably, learning experiences that reduce the need to always attend a face-to- 
face class. 

As regards students’ perceptions of the useRilness of online learning to Academic Writing, it is instructive 
that the large majority had mixed perceptions. This implies that all is not lost and herein lies an opportunity to so 
design the integration of online learning into the Academic Writing modules that those who are not sure of the 
efficacy of online learning in Academic Writing may not join the ranks of those whose perceptions are negative. 

The finding that the most preferred learning modality is face-to-face was not totally unexpected. 
Unpublished research commissioned by the School of Humanities & Social Sciences, University of Technology, 
Jamaica, indicates a preference for face- to-face classes among high school students. Black-Hughes, et al, (2008) 
also found that students preferred teacher contact via face-to-face classes. Culture also affects students’ perceptions 
and attitudes, which influence behaviour (Lee, 201 1). Jamaican students are known to be social learners. That is, 
many do not view favourably learning situations with any hint of isolation from peers or the instructor. There is also 
the cultural attitude of ‘sticking to the familiar’. The traditional learning modality in Jamaican educational 
institutions is face-to-face, and so students might be reluctant to make any shift from the familiar. 

Implications & Conclusion 

The foregoing findings have implications for the utilization of web-assisted, blended and distance online 
learning modalities in Academic Writing, to which the School of Humanities and Social Science is committed. 
Students still show a preference for face-to-face classes and there is poor attitude to online learning among most of 
the full time students surveyed. Nevertheless, the better attitude to online learning among working students can be 
capitalized upon and efforts can be made to improve the mixed perceptions students have with regard to the 
usefulness of online learning to Academic Writing. 

There are limitations to this study such as the small sample size and the non-random selection of 
participants, but the facts unearthed cannot be ignored. Statistical analysis at the item level reveal that many of the 
students are not satisfied with their access to computers and the internet at the university. Others also do not own or 
have easy access to computers and the internet outside of the university. 

It is recommended that qualitative research be done to fill the gaps leR by this and similar studies on students’ 
readiness, perceptions and attitudes to online learning at this university. Student readiness for online learning has 
remained largely unexplored at this university. Areas such as the impact of culture on student learning modality 
preference also need to be explored. 
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Abstract 

In the absence of state educator certification in instructional technology, and state-wide staffing 
requirements for instructional technology specialists, media specialists may be playing an increasingly larger role in 
instructional technology support and focusing less on other vital media specialist responsibilities. The purpose of 
this quantitative survey study was to examine the role of media specialists with respect to instructional technology. 

A deeper understanding of the role of media specialists with respect to instructional technology may provide insight 
into determining a need for instructional technology certification and support in k-12 schools. Practicing media 
specialists’ perceived use, and perceived ideal use, of instructional technology specialist and media specialist job 
competencies were examined. The data revealed an overall difference among the four dependent variables (a) 
perceived current use of media specialist competencies, (b) perceived ideal use of media specialist competencies, (c) 
perceived current use of instructional technology specialist competencies, and (d) perceived ideal use of 
instructional technology specialist competencies. Within-subjects contrasts revealed significant pairwise differences 
among all the variables except the comparison of the use of media specialist competencies and the use of 
instructional technology specialist competencies. These findings suggest that in the absence of consistently staffed, 
certified instmctional technology specialists, media specialists are playing an increasingly larger role in instructional 
technology support and focusing less on other essential media specialist roles and responsibilities. 

Introduction 

Without a Georgia Educator Certificate in instructional technology and statewide staffing regulations for 
instructional technology specialists, there is a lack of standardization in the qualifications and staffing of P-12 
instructional technology specialists in Georgia public schools. On the contrary, the Georgia Professional Standards 
Commission and the Georgia Department of Education provide standardized support for school library media 
programs through the certification and staffing of certified media specialists in all public schools statewide. A 
greater understanding of the role of media specialists with respect to instructional technology could provide insight 
into establishing a need for the support of consistently staffed instructional technology specialists in Georgia public 
schools. Practicing media specialists’ perceived use, and perceived ideal use, of instructional technology specialist 
job competencies, as defined by the International Society for Technology in Education (ISTE) 2001 Educational 
Computing and Technology Standards for Technology Facilitation Initial Endorsement, were examined. The use of, 
and perceived ideal use of, media specialist job competencies, as defined by the 2010 American Association of 
School Libraries (AASL) Standards for Initial Preparation of School Librarians were also examined. The following 
questions guided this research design and data analysis: 

1. Is there a difference between participants’ perceptions of their current use and their perceptions of ideal 
use of media specialist competencies? 

2. Is there a between participants’ perceptions of their current use and their perceptions of ideal use of 
instructional technology specialist competencies? 

3. Is there a difference between participants’ perceptions of their current use of media specialist 
competencies and their perceptions of their current use of instructional technology specialist 
competencies? 

4. Is there a difference between participants’ perceptions of ideal use of media specialist competencies 
and their perceptions of ideal use of instructional technology specialist competencies? 
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Literature Review 


Instructional Technology Program Standards 

There were two sets of National Council for Accreditation of Teacher Education (NCATE) Specialized 
Professional Associations’ (SPA) standards related to instructional technology. These standards were developed by 
the Association for Educational Communications and Technology (AECT) and the International Society for 
Technology in Education (ISTE). In March 2011, AECT notified NCATE that it would withdraw as an NCATE 
SPA effective June 30, 2011. AECT previously reviewed two types of programs in the field of instructional 
technology, Initial School Media and Educational Technology’ Specialist (I-SMETS) and Advanced School Media 
and Educational Technology’ Specialist (A-SMETS). Currently there are programs at 32 colleges and universities in 
21 states that were approved by the AECT for adhering to the standards of program preparation (NCATE, 2010). 

The ISTE standards for teacher preparation in the field of technology are also divided into two programs: Initial and 
Advanced. In 1996 NCATE approved ISTE’s original program standards, Program for Initial Preparation of 
Teachers of: Educational Computing and Technology’ Literacy and Program for Advanced Preparation of Teachers 
of: Educational Computing and Technology’ Leadership Endorsement. In 2001 the original ISTE/NCATE standards 
{Technology’ Facilitation Initial Endorsement and Technology’ Leadership Advanced Program) were revised and 
updated. New ISTE standards (Standards for Technology’ Coach and Director ) were approved by NCATE in Fall 
2012. Currently there are 12 states with colleges and universities with nationally recognized Technology Facilitation 
programs (NCATE, 2010). In these 12 states, there are 29 colleges or universities, which offer a total of 31 
programs. Furthermore there are seven states with colleges and universities with nationally recognized Technology 
Leadership programs. In these seven states there are 1 1 colleges or universities that offer a total of 1 1 programs. 

Library’ Media Program Standards 

Like instructional technology, there were two sets of Specialized Professional Association (SPA) NCATE 
approved standards related to programs that prepare school media specialists: AECT Standards for the Accreditation 
of School Media Specialist and Educational Technology’ Specialist Programs, and the American Association of 
Libraries (ALA) / American Association of School Libraries (AASL) Standards for Initial Programs for School 
Library’ Media Specialist Preparation. In March 2011, AECT notified NCATE that it would withdraw as an 
NCATE SPA effective June 30, 201 1. Currently there are six programs, at five colleges or universities, in five states 
with nationally recognized AECT School Media Specialist programs (NCATE, 2010). AASL and NCATE united in 
1989 to develop standards for accrediting school media specialist preparation programs, Curriculum Folio 
Guidelines for the NCATE Review Process: School Library Media Specialist Basic Preparation. Revised standards 
were accepted in 1993 based on Information Power: Guidelines for School Library’ Media Programs. In 1998 
updated guidelines for school library media programs were released, Information Power: Building Partnership and 
the new AASL Standards for Initial Programs for School Library’ Media Specialist Preparation were approved by 
NCATE in 2002. Due to significant changes in school library media programs that have increased the importance of 
technology, AASL introduced a new set of guidelines for school library media programs in 2009, Empowering 
Learners: Guidelines for School Library’ Media Programs. Based on the new guidelines the AASL revised their 
NCATE SPA standards in October 2010. Currently there are 22 states with colleges and universities with nationally 
recognized AASL school librarianship education programs. In these 22 states, there are 37 colleges, or universities 
that offer AASL recognized programs. Furthermore there are seven states with colleges and universities with 
nationally recognized programs. 

Media Specialist Role 

Empowering Learners: Guidelines for School Library Media Programs center on developing a flexible 
learning environment with the goal of creating successful learners accomplished in multiple literacies (AASL, 

2009). The guidelines require library media programs to meet the requirements of the changing school library 
environment as directed by the Standards for the 21st-Century’ Learner and Standards for the 21st-Century Learner 
in Action (AASL, 2009). The rapid and massive influx of technology in education has significantly impacted the 
skill set media specialists need to be successful. Technology integration hinges on the support of media specialists 
(Forrest, 1993). Media specialists spend a significant proportion of their day on tasks related to the diffusion of 
information technology (Forest 1993; McIntosh 1994). As a result, media specialist preparation programs are 
adapting by offering more technology courses. According to a survey distributed to library science faculty members 
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at ALA accredited programs, 71% of ALA accredited programs offer technology-related courses (Harada, 1996). 
Callison and Tilley (1999) found that changes in course offerings in 25 ALA accredited programs for school media 
specialists included more attention to multimedia, Web site, and video production over the course of previous five 
years, and less on resources for children and youth and library administration. McCoy (2001) used a modified 
version of Woodruffs (1994) competency survey with practicing school media specialists to determine the job 
competencies that they value most. The results showed that the respondents placed administration, information 
access and delivery, and collection development as the focus of the school library media program. The respondents 
showed a high degree of general interest in technology integration and implementation, and they showed less 
interest in specific computer applications. 

Instructional Technology’ Support 

Providing teachers and students access to technology, as well as providing quality professional 
development for teachers to integrate technology into teaching and learning, is essential for students to acquire 
technology competencies required in today’s job market (U.S. Department of Education, 2000). Research has 
demonstrated that increased technology professional development and instructional support positively impacts 
technology integration in K-12 education. DiBenedetto (2005) reported that technology trained teachers exhibit 
positive attitudes toward using technology and use more technology than teachers who did not have technology 
training. O’Dwyer, Russel, and Bebell (2004) reported that low teacher perception of support and inadequate 
professional development, negatively impact technology integration. Wahl (2000) recommends 70% of technology 
funds be spent on professional development with 30% spent on technology infrastructure. Ronnkvist, Dexter, and 
Anderson (20002) found that inadequate technology support impedes the effective integration of technology into 
classroom learning and found that technology specialists were essential in providing both support and pressure for 
change. Abbott (2003) found that 62% of teachers reported ‘not enough’ or ‘barely enough’ technology support 
personnel are available, and 64% reported not enough time available from technology support personnel to deliver 
technology professional development. May (2000) and Davis (2002) found that teachers who receive mentoring 
and/or follow up support for technology training, integrate technology more often than teachers without support. 
Carlson (2002) reports that teacher training is a crucial factor for integrating technology to improve student 
performance. Carlson promotes teacher training that includes ongoing pedagogical support to help teachers tackle 
challenges of teaching. Hofer, Chamberlin, and Scott (2004) describe how technology integration specialists can 
serve as change agents supporting curriculum and pedagogy renewal. Gahala (2001) proposes that all schools have a 
site based technology specialist. Bernal (2001) examined leadership factors that influence the implementation of 
technology and found that access to, and support for, technology in the school are the most influential factors in 
successful technology integration. Cuban (2001) contended that support is essential for teachers to apply their 
recently acquired skills and cultivate their on-going technology development. “The infrastructure of technology 
support and professional development would need to be redesigned and made responsive to the organizational 
incentives and workplace constraints teachers’ face” (Cuban, 2001, p. 183). 

Shoffner (2000) discussed the results of a national survey conducted to determine how instructional 
technology specialists are staffed and certified across the U.S. Results indicated that most states did not have 
certification programs in instructional technology and the instructional technology positions are staffed in a wide 
variety of ways. Shoffner reported that only seven states (Pennsylvania, Massachusetts, Maine, New Hampshire, 
Wisconsin, North Carolina, and New Mexico) had licensing rules for instructional technology specialists and one 
state, Vermont, had a proposal for an instructional technology coordinator license. In New Mexico and Maine, 
endorsements were available and Vermont had a proposed endorsement. The other five states (Pennsylvania, 
Massachusetts, New Hampshire, Wisconsin, North Carolina,) had full licenses available. Goetzel (2008) discusses 
the results of a 2005 follow up study to Shoffner (2000). Department of Education websites for all 50 states were 
examined to determine if change had taken place since 2000. In 2005 an additional eight states had licensing 
requirements for instructional technology positions. Endorsements to existing teaching certificates at that time were 
offered in 12 states. New York was the only state that had added a full initial certification. In addition, 10 states 
(Minnesota, New Jersey, Virginia, Connecticut, Pennsylvania, Delaware, California, Missouri, Texas, and 
Minnesota) had instructional technology certificates available from colleges of education. These certificates are not 
licenses or endorsements offered by the states licensing authority. They are certificates offered by colleges of 
education either in conjunction with existing degree programs in instructional technology or as standalone 
certificates not associated with degree programs. 
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Job Competencies and Analysis 

Job competencies are defined as a measurable pattern of knowledge, skills, abilities, behaviors, and other 
characteristics that an individual needs to perform work roles or occupational functions successfully (U.S. Office of 
Personnel Management, 2011). The International Board of Standards for Training Performance and Instruction 
(IBSTPI), defines a competency as “an integrated set of skills, knowledge, and attitudes that enables one to 
effectively perform the activities of a given occupation or function to the standards expected” (International Board 
of Standards for Training Performance and Instruction [IBSTPI], 2010, para 2). Organizations like AASL, ISTE, 
and AECT provide competency-based standards for accrediting academic programs in instructional technology and 
library media. Instructional technology and library media competencies as defined by the performance indicators in 
the ISTE and AASL standards are used for the purpose of this study. According to Rothwell & Kazanas (2008) in 
Mastering the Instructional Design Process , job analysis clarifies what activities personnel should be responsible for 
and the outcomes they should be attaining. Job analysis could also divulge impediments to performance that go 
beyond the control of personnel and require counteractive action by management. Additionally, the outcome of job 
analysis can be a beginning point for more comprehensive task or content analysis. (Rothwell & Kazanas, 2008). 

Methodology 


Research Design 

A quantitative descriptive and comparative research design was utilized in this study. This study employs a 
quantitative research framework that uses postpositivist claims for developing knowledge. The strategy of inquiry 
used in the study was an independent cross-sectional survey study. The form of data collection used was by way of 
the Internet through a web-based survey. 

Sample, Population, and Participants 

The population targeted for the study is practicing certified media specialists (K-12) in one urban school 
district in Georgia. The sampling design for the study was single stage convenience design with no stratification or 
clustering. The decision was based upon access to specific individuals in the population. With email addresses of the 
members of the population, they could be sampled directly. An a-priori power analysis was conducted to determine 
the number of participants required to detect a medium effect size (f = .25) with power = .80 for a repeated measures 
ANOVA (analysis of variance) conducted at = .05. The power analysis suggested that 45 individuals were 
needed to achieve a power of .80 given these parameters. The power analysis was conducted with the statistical 
software G*Power 3.1.0 (Faul, Erdfelder, Lang, & Buchner 2007). The survey was distributed to all 93 members of 
the population to maximize response rate. 

Instrumentation 

The instrumentation used in this study was an original questionnaire designed and created for this study by 
the researcher. The survey instrument was based on two existing surveys with an identical Likert-type scale from 
previous established research by Woodruff (1994) and McCoy (2001). Furthermore, a job competency survey for 
assistant principals with the same format was used as a model (Madden, 2008). The instrument was divided into two 
parts. Part one consists of seven items to collect demographic characteristics of the participants to validate the 
sample and to create demographic profde of the participants. Part two consisted of 76 statements that represented 
job competencies used by instructional technology specialists (33) and media specialists (43) as defined by the 
performance indicators in the 2001 ISTE and 2011 AASL, NCATE SPA standards. Participants were also provided 
an opportunity to offer additional information relevant to the study in an open-ended item. The items in Part Two 
were rated using a Likert-type scale to assess perceptions regarding; the extent participants use instructional 
technology specialist and media specialist job competencies, and perceptions regarding the extent participants feel 
instructional technology specialist and media specialist job competencies should ideally be used. For each 
competency participants first selected the number that reflects the extent to which they feel they are currently using 
the competency as a Media Specialist in their present position. Then for each competency, participants selected the 
number that reflects the extent to which they perceive they should ideally use the competency as a Media Specialist. 
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Data Collection Procedure 


The survey was administered on the Internet through Survey Monkey; a Portland, Oregon based online 
survey company formed in 1999. The survey was distributed using the web deployment option in Survey Monkey to 
create a URL sent in an email via the school districts’ electronic mail distribution list for media specialists. To 
obtain a greater response rate, the Dillman (2007) tailored design method was used to collect the data. After 13 days, 
access to the survey was closed. 


Results 


Reliability 

Cronbach’s alphas (Table 1) were calculated to determine the level of internal consistency reliability of the 
four subscales (Whitley, 2002). All of the subscales demonstrated sufficient levels of internal consistency reliability. 
Subscale reliability ranged from .962 (instructional technology current use) to .969 (media specialist ideal use) for 
these data. 

Table 1 


Cronbach 's Alphas for Survey Subscales 


Scale 

n of 

Cronbaeh's 


items 

Alpha 

Extent media specialist competencies are used in their position 

43 

.965 

Extent media specialist competencies should ideally be used in 
their position 

43 

.969 

Extent instructional technology specialist competencies are used 
ill their position 

33 

.962 

Extent instructional technology specialist competencies should 
ideally be used in their position 

33 

.967 


Hypothesis Testing 

A repeated-measures ANOVA (analysis of variance) was conducted to address the study hypotheses. 
Histograms of the participants’ perceptions of their current use and their perceptions of the ideal use of media 
specialist competencies are displayed in Figures 1 and 2, respectively. The histograms of the participants’ 
perceptions of their current use and their perceptions of the ideal use of instructional technology specialist 
competencies are displayed in Figures 3 and 4, respectively. The histograms for the use of media specialist 
competencies and the use of instructional technology specialist competencies indicated that the distributions were 
approximately normal. However, the histograms for the ideal use of media specialist competencies and the ideal use 
of technology specialist competencies indicated negatively skewed distributions (Howell, 2004). This indicates that 
the extreme (i.e., unusual) scores were on the low end of the scale for both variables. Mauchly’s test of sphericity 
was significant, indicating inequality of error variances and covariances. Therefore, a Greenhouse-Geisser correction 
was used to adjust the degrees of freedom (Stevens, 2002). 
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Figure 1. Distribution of Current Use of Media Specialist Competencies 



Figure 2. Distribution of Ideal Use of Media Specialist Competencies 



Figure 3. Distribution of Current Use of Instructional Technology Specialist Competencies 
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Instructional Technology Specialist Competencies Ideal Use 


Figure 4. Distribution of Ideal Use of Instructional Technology Specialist Competencies 

The means and standard deviations of each variable are listed in Table 2. The AN OVA (Table 3) revealed 
an overall difference among the dependent variables, F (1.34, 72.36) = 72.13, p< .01 ( 2 = .57, power = 1.00). 
Within-subjects contrasts (Table 4) were conducted to further examine the significant ANOVA effect (Howell, 
2004). The tests revealed significant pairwise differences among all the variables except the comparison of the use 
of media specialist competencies (M = 3.80, SD = 0.62) and the use of instructional technology specialist 
competencies (M = 3.72, SD = 0.66), /’(l, 54) = 3.64, p > .05 ( 2 = .06, power = .47). This indicates that there was 
not a significant difference between their usages of the two core competencies. However, the participants scored 
significantly higher on their perceptions of the ideal use (M = 4.58, SD = 0.43) of media specialist competencies 
compared to their current use (M = 3.80, SD = 0.62) of media specialist competencies. The tests also revealed that 
the participants scored significantly higher on their perception of the ideal use (M = 4.49, SD = 0.50) of instructional 
technology specialist competencies compared to their current use (M = 3.72, SD = 0.66) of instructional technology 
specialist competencies. Lastly, the tests also showed that the participants scored significantly higher on their 
perception of the ideal use (M = 4.58, SD = 0.43) of media specialist competencies compared to their perceived 
ideal use (M = 4.49, SD = 0.50) of instructional technology specialist competencies. 

Table 2 

Means & Standard Deviations of Current & Ideal Competency Use 


Variable 

n 

M 

SD 

Current Use of Media Specialist Competencies 

55 

3.80 

0.62 

Ideal Use of Media Specialist Competencies 

55 

4.58 

0.43 

Current Use of Instructional Technology Specialist Competencies 

55 

3.72 

0.66 

Ideal Use of Instructional Technology Specialist Competencies 

55 

4.49 

0.50 
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Table 3 


Repeated-Measures ANOVA on Current & Ideal Competency Use 


Source 

ss 

df 

MS 

F 

P 

II 2 

Power 

Competency 

33.14 

1.34 

24.73 

72.13 

.000 

.57 

1.00 

Error 

24.81 

72.36 

0.34 





Tabic 4 








Repeated-M easu res 

Post Hoc C 

omparisons 






Source 




df F 

P 

T)' Power 


Media Competency 

Use vs. Media Competency Ideal Use 

1 84.57 

.000 

.61 1.00 



54 (0.39) 

Media Competency Use vs. Technology Competency Use 1 3.64 .062 .06 .47 

54 (0.11) 

Media Competency Ideal Use vs. Technology 1 13.39 .001 .20 .95 

Competency Ideal Use 

54 (0.03) 

Technology Competency Use vs. Technology 1 75.67 .000 .58 1.00 

Competency Ideal Use 

54 (0.43) 


Mote. Number in parentheses represents MSE for corresponding error term. 

Conclusions 

The data analysis revealed an overall difference among the four dependent variables. The tests also 
revealed significant pairwise differences among all the variables except the perceived current use of both core 
competencies. The data analysis indicated that perceived current use of media specialist competencies is 
significantly less than perceived ideal use. This suggests the media specialists perceived they were unable to take 
fill advantage of the media specialist competencies. This indicates barriers may exist which prevent the media 
specialists from using the media specialist competencies to the ideal extent. The analysis also showed that perceived 
current use of instructional technology competencies was significantly less than perceived ideal use. This indicates 
barriers may exist which prevent the media specialists from using the instructional technology competencies to the 
ideal extent. Additionally, the results indicated that the perceived current use of media specialist competencies is not 
significantly different from the perceived current use of instructional technology competencies. This indicates there 
was no significant difference in the extent the media specialists perceive they are currently using both competencies. 
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This suggests barriers could exist which prevent the media specialists from using their core media specialist 
competencies to a significant extent in relation to their use of both sets of competencies. Moreover, the results also 
indicate that the media specialists’ perceived ideal use of instructional technology competencies was significantly 
less than their perceived ideal use of media specialist competencies. This showed that although there is no 
significant difference in the extent the media specialists perceive they are currently using both competencies, the 
media specialists perceive they should ideally be using the media specialist competencies to a greater extent than the 
instructional technology competencies. This suggests the media specialists believe the use of their core media 
specialist competencies to be of greater importance than the use of instructional technology competencies. 
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Introduction 

The rise of nontraditional, practitioner-based doctoral programs continues to touch the study of Educational 
Technology and related fields. Although one purpose of these contemporary programs is to prepare advanced 
practitioners to bring leadership and scholarship into their respective places of employment, an unintended 
consequence may be the rise of a second class of doctoral scholars. Due to the different focus of these programs, 
graduates may not be as competitive and their counterparts from traditional programs. Specifically, they may be less 
competitive when seeking positions in academia. To help remediate this issue, students enrolled in practitioner- 
focused programs may have to make conscious efforts to become more competitive in and prepared for traditional 
academic role. 

To further explore these issues, a brief survey was specifically developed to investigate the roles of students 
enrolled in online versus traditional graduate programs in Educational Technology and related fields. The survey 
looked at various roles and professional activities including: cohort leadership, involvement in professional 
associations, publishing, and presenting at conferences. 

Student Involvement 

As enrollment increases and technology improves, students enrolled in online graduate programs in Educational 
Technology and related fields are presented with opportunities for involvement but also unique challenges as 
compared to traditional graduate students. Traditional programs offer an array of well-established and widely- 
supported involvement opportunities including: graduate and research assistantships, teaching assistantships, 
practica experiences and internships, involvement in professional organizations, and involvement in university- 
specific organizations and initiatives. As such, students in online programs may lack these kinds of experiences and, 
in turn, be less competitive than their traditional graduate student counterparts. 

Traditional vs. Online 

Existing research in this area appears to focus more on online graduate students’ perceptions of their experiences 
rather than on the active roles students play within their respective programs. For example, Picciano (2002) found a 
strong relationship between students’ perceptions of both the quality and quantity of their interactions and their 
perceptions of their performance in online coursework. The limited research on the roles of online graduate students 
suggests a ripe avenue for further inquiry. 

Relative to their peers in traditional graduate programs, students in online programs may have fewer ties to their 
universities’ main campuses. Students enrolled in some online programs, for example, may reside in other parts of 
the country and never be required to step foot on to the physical campus. As such, many campus-specific 
involvement opportunities may not be available to online students. Moreover, students in online programs may not 
have access to traditional assistantships, practicum experiences, or internships. Because of this, online graduate 
students may rely on networking on social media such as Facebook, Twitter, and Google+ as means of 
communication and staying connected with fellow cohort members and professors (Roblyer et ah, 2010). 
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Seeking Leadership Opportunities 


Students in online graduate programs may be able to complete their programs’ coursework without requirements 
for additional involvement. For students who want to become more actively involved in their programs and fields of 
study, though, a proactive attitude as well as a certain level of creativity may be necessary. These students’ efforts 
have the potential to not only help them become involved in otherwise inaccessible, traditional opportunities, but 
also forge new opportunities that are unique to the online graduate experience. 

In his series of essays titled, “Write Your Dissertation First and Other Essays on a Graduate Education”, David 
Merrill emphasizes the importance of graduate students refusing to take no for an answer when searching for 
involvement opportunities as well as the need to be open to creative exploitation as a means of gaining experience. 
In this spirit, students enrolled in online programs may be required to think outside of the box and independently 
seek out opportunities for leadership and involvement. For example, students may be forced to make the 
professional contacts necessary to set up their own internships, join and become involved in state and national 
professional organizations such as the AECT and its Graduate Student Assembly (GSA), or independently locate 
funding to attend national professional conventions and conferences. 

In order to standout in their online programs, graduate students may also seek out informal leadership 
opportunities within their respective cohorts by striving to be seen as both responsible leaders and highly 
conscientious students. By doing so, students may be asked to collaborate with their professors on special projects 
such as coauthoring scholarly articles, presenting at conferences, and providing input related to other initiatives such 
as strategic planning. 


The Survey: Skimming the Surface 

At the time of this report, 37 graduate students had responded to a brief survey conducted specifically for this paper 
and presentation. The survey consisted of the following items: 

1. What degree are you currently pursuing? 

• Masters 

• Ed.D 

• Ph.D. 

• Other: 

2. What is your degree program? 

• Educational Technology 

• Instructional Technology 

• Other: 

3. Which best describes your graduate program? 

• Traditional 

• Fully-Online 

• Hybrid 

• Other: 

4. If you answered "online" or "hybrid", what percentage of your classes would you estimate are offered online? 

5. Have you participated in any of the following opportunities so far in your current graduate program? Check all 
that apply: 

• Published an article in a scholarly journal 

• Won or was nominated for a university or departmental awards 

• Joined a professional organization in the field (i.e., AECT, AERA, ISTE, etc.) 

• Attended a professional conference 

• Held an office or a leadership position in a professional organization in the field (i.e., AECT, ISTE, etc. 


62 



• Presented at a national or international conference 

• Presented in an online or virtual conference 

• Served as a Teaching Assistant 

• Served as a Research Assistant 

• Other: 

6. Overall, how would you describe your professors' or advisors' roles in making you aware of the aforementioned 
opportunities? 

7. Do you feel that your choice of graduate program has impacted your ability to participate in the kinds of scholarly 
activities described above? If yes, please describe below: 

8. Are you satisfied with your current access to the kinds of scholarly opportunities described above? 

• Yes 

• No 


Themes and Takeaways 

A total of 37 graduate students responded to the survey. One student was eliminated, because their graduate 
program was not in Educational Technology or a related field. Related fields included: Instructional Technology, 
Instructional Systems, Instructional Design, Educational Technology Leadership, and Instructional Design and 
Development. Respondents were recruited via email and the Facebook, and they accessed the survey via a Google 
Forms link. Of the respondents, only 3 students described their graduate program as traditional, whereas 1 1 
described their program as “Hybrid” and 23 described their program as “Fully-Online”. Although the “traditional” 
sample was comparatively small, all three respondents reported that they were satisfied with their current access to 
the kinds of scholarly and leadership opportunities described in survey question 5. With the exception of two 
students who did not respond to question 5, all of the students who identified their program as “Hybrid” indicated 
that they were satisfied with their current access to scholarly and leadership opportunities. With the exception of two 
students who did not respond to question 5, the majority of respondents reported that they were satisfied with their 
current access to scholarly and leadership opportunities, but 20% indicated that they were not. 

The majority of respondents indicated that their professors made them aware of and encouraged them to take 
advantage of the kinds of scholarly and leadership opportunities described in question 5. Of the students who 
indicated that they were dissatisfied with their current access to scholarly and leadership opportunities and listed 
several contributing factors including: a) being at the beginning of their graduate coursework, being restricted by the 
nontraditional, online format, and their professors staying occupied with their own work load and research interests. 

The results of this brief survey were inconclusive but highlighted directions for future inquiries and ways in which 
the research can tighten up and refocus the questions. On goal of futures studies will be to include more questions 
and recruit more respondents — particularly those enrolled in traditional programs. Future studies could look at 
variables such as gender, time left until graduation, and personality factors. 

In Conclusion: Future Directions 

More data needs to be gathered in order to clarify the relationship between student involvement in scholarship 
and leadership opportunities in relation to their graduate program’s instructional delivery methods. Professors in 
online graduate programs could use their students in innovative ways to further promote leadership and 
involvement. Doing so may help online programs to elevate their reputations by producing stronger, more highly 
qualified graduates. They may also do this by involving standout students in program development activities, 
strategic planning, and the faculty interview process. 

In conclusion, students in online graduate programs in Educational Technology and related fields may have to 
think outside of the box in order to create their own opportunities for involvement and leadership. Students who 
meet this challenge may, in turn, also break the ground for future online students to be vibrant leaders in the field. 
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Abstract 

This study investigated the students’ perspectives of factors that impact on students’ low motivation 
toward online discussions and further explore how students’ individual differences influence their perspectives. The 
findings of this study indicated that students perceived discussion topics, teachers’ expectations, and online learning 
community as the main factors to influence the motivation in online discussions. Moreover, students’ perspectives 
on the motivation toward online discussions were also influenced by their online learning experiences and 
knowledge of computers and technology. 


Introduction 

To enhance online students’ communication, asynchronous online discussion has been perceived as an 
important component in online teaching and learning to facilitate computer-mediated communication (CMC) and 
generate student-to-student and student-to-instructor interaction. Few previous studies have investigated the factors 
that impact aspects of online discussions quality. Spatariu, Quinn, and Hartley (2007) reviewed prior research and 
implied two main factors to influence the quality of online discussion: instructional interventions and learner 
characteristics. Du, Zhang, Olinzock, and Adams (2008) investigated graduate students’ perceptions of the factors 
that contribute to the quality of online discussion by conducting a qualitative case study. Their study found the key 
factors that influence the quality of online discussions from students’ perspectives were: manners of response, size 
of the group, and the topic of discussion. 

However, little research has delved into the specific reasons why students have low motivation in online 
discussions and how students’ individual differences influence their perceptions on low motivation of online 
discussions. Therefore, the purpose of this study is to investigate the factors that influence students’ low motivation 
in online discussions from students’ perspectives and further explore the relationship between students’ individual 
differences and their perspectives. The findings of this study shed light on the better understanding the factors that 
influence the quality of online discussions for online courses designers or teachers. 

Research Methodology 


Participants 

There were 171 undergraduate students enrolled an online course entitled, “Computing and Information 
Technology” during the semester. It is an introductory course to computers and technology. Online discussion was 
an important learning activity in this course and was accounted for 10% of students’ final grades. 

Procedure 

During the semester, all students who were enrolled in the course were required to participate in five 
asynchronous online discussions in Blackboard WebCT. Each discussion was related to one chapter in the course 
and lasted a week. When discussions were finished, students were required to complete an online survey. The 
survey included two parts: Part I: Background Information and Part II: Why students have low motivations in online 
discussion. Part II was composed of three main aspects: reasons of students, reasons of teachers, and reasons of 
technology. Three aspects had ten, seven, and seven items, respectively. Likert scales were used in the survey to 
ask students to answer questions by choosing (1) strongly disagree, (2) disagree, (3) undecided, (4) agree, or (5) 
strongly agree. Finally, there were 139 students completed and submitted the survey online. The reliability 
coefficient of Cronbach's alpha for this survey is .88. 
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Results 


The mean score for each survey aspect and item was calculated. The mean scores for three aspects (reasons 
of students, reasons of teachers, and reasons of technology) were: M = 3.46, M = 3.12, and M = 3.12, respectively. 
From students’ perspectives, the most possible reasons why some students have low motivation in online 
discussions are “students may have no interests in discussion topics” (M = 3.92) and “Students may worry about 
what they post are not what the instructors expect” (M = 3.91). Some other highly possible reasons are “Students 
may be shy to express their own opinions” (M = 3.71) and “Students may feel online discussion cannot enhance 
their learning in the online courses” (M = 3.61). 

A series of correlation analyses conducted to examine how students’ individual differences influence their 
perspectives of the low motivation in online discussions. The results found that students’ experiences of online 
learning was negatively related to students’ perspectives of reasons of students on low motivation in online 
discussions, r = -.112, p < .005, reasons of teachers, r = -.181,/>< .005, and reasons of technology, r = -.188, 

< .005. Moreover, students’ different knowledge of computers and technology was positively related to students’ 
perspectives of reasons of students on low motivation in online discussions, r = .207, p < .005. However, students’ 
different school years, ages, and levels of English language had no relationship with their perspectives of why 
students had low motivation in online discussions. 

Discussion and Conclusion 

This study revealed that from students’ perspectives, interesting and valuable discussion topics, clear 
teachers’ expectations, and effective learning community were the main factors to influence their motivation in 
online discussions. If the discussion topics are not interesting, students will feel boring and lose motivations to 
discuss. Moreover, the discussion topics should be closely connected to the course content and motivate students’ 
debating or critical thinking. The primary aim is to attract students’ interests and help them construct knowledge 
and improve their higher order thinking skills. Clear discussion policies or guidelines should also be designed to 
show students how important they attend the online discussion and freely express their own opinions in online 
discussions. Both quantitative and qualitative expectations need to be explained to help students exactly know what 
they should do to achieve the discussion requirements. Building up an effective online learning community is also 
important to motivate students. A free and open discussion environment should be established to encourage all 
students to feel free to voice their opinions. 

Another interesting finding in this study was students’ experiences of online learning were negatively 
associated to students’ perspectives on low motivation in online discussions. When students have more online 
learning experiences, they more positively attend online discussions and do not think many students have low 
motivations in online discussions. This result indicates that online courses designers or teachers need to pay 
attention on students’ prior experiences of online learning and online discussions. Therefore, practice online 
discussion can be designed to let students get familiar with online discussion at the beginning of the semester. 
Moreover, students’ perspectives on low motivation toward online discussions were also influenced by their 
knowledge of computers and technology. When students have more knowledge of computers and technology, they 
tend to perceive the student self as the key reason, rather than the teacher or the technology to lead to the low 
motivation toward online discussions. This result was consistent with previous research. Krentler and Willis-Flurry 
(2005) indicated that students’ use of technology had a significant main effect on students’ learning. Online 
discussion is a complex communication process. Many factors interact with each other to impact on the motivation 
and the quality of online discussions during this process. Online courses teachers or designers should overall 
consider these factors and try to design an effective online discussion to fit more students’ characteristics and needs. 

The limitation of this study lies on the specific course. The survey data only came from the specific 
participants who took the basic computer skill entry course. Future studies could validate the results of this study or 
investigate more reasons why students have low motivation in online discussions and how students’ individual 
differences influent by collecting data from more different online students. 
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Higher education institutions are increasingly using online advising to recruit and retain students. 
These electronic advising provide an online alternative for students to access and share information and 
build a community with faculty and other students (Waldner, McDaniel, Esteves, & Anderson, 2013). 
Online advising can also be used to improve the communication and mentoring relationship between the 
department advisor and the students. The Virginia Community College System’s guidelines for online 
advising include easy access and ease of use, accessibility at any time, equal to on-campus advising, 
privacy and security, opportunity for personal one-on-one advising, allow students access to own records, 
and an interactive calendar that links to forms, information, and deadlines. These best practice methods 
for online advising can helps students who are at a distance and/or work full time. 

The current research project is collaboration among an IDT graduate student, the IDT graduate 
advisor, and the IDT department chair. They created an online graduate advising presence for the 
department because most of the graduate students in the Instructional Design and Technology Department 
do not live in the same area that the university is located. The students typically take a combination of 
online courses and weekend courses. Since the students are located at a distance and/or work full time, it 
is difficult for them to meet face-to-face with the department’s graduate advisor during regular office 
hours. The department is also seeing an increase of out-of-state students whose degree is completely 
online. 

During the first phase of development, an online advising website was developed for graduate 
students in the Instructional Design and Technology Department using the university’s course 
management system. The website supplements the Graduate Studies’ and department’s website. The 
website provides students samples of portfolios, applied projects, and theses for online students who do 
not have access to printed versions. The webpage includes detailed information on the steps to 
completing the degree, examples of completed required forms such as sample degree plans, request for 
exit option committees, and examples of proposals and requests for exit option courses. 

The course area allows the graduate advisor to post internal mail messages to individual students 
or all the enrolled graduate students. The mail area is easier for the advisor and the students to search past 
messages than on a regular email account. The online course area has successfully been used by the 
graduate advisor, department chaiiperson, and the departmental secretary to convey information and to 
interact in a timely manner with students. The discussion forums are broken into the following 
categories: General Information, admissions, course selection, weekend academies, and exit options. The 
general information category contains discussion forums on general questions about the graduate 
program, IDT-related events and publications, and a general student-to-student lounge area. The other 
discussion forums contain areas where the students can ask questions about specific parts of the program 
and receive answers and comments from the advisor and from other students. 
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By creating an online advising website for graduate students in the department of Instructional 
Design and Technology, we wanted to give students more autonomy and create more efficiency for the 
department graduate coordinator. Thus, in the future when the graduate coordinator position is bestowed 
on another faculty member, there will be continuity in the information presented. 

The online graduate advising area prototype was opened to the students in 2009. The discussion 
area has already generated over 175 student-initiated posts. Over 100 internal mail messages have been 
sent through the advising area by students to the graduate advisor. The online chat is very popular since 
students can get questions immediately answered by the graduate advisor when she is online. This has 
also cut down on the number of long distance phone calls that online students make to the graduate 
advisor. 

The researchers are currently evaluating the use of the advising area by the students. Student 
discussion posts, mail messages, general student activities, and tool usage are being considered in order to 
determine the next revision of the pages. Faculty and students will be surveyed on their perceptions of 
usefulness of the new advising area and on what they would like to see added. 

The second phase of the program included updating and revising the advising area, plus a move 
to a new course management system when the university changed from Blackboard/WebCT to 
Desire2Leam in 2012. 

By this time there were many student-initiated discussions regarding weekend academy classes 
and travel/hotel sharing, books, suggestions for hardware/software, instructors, class content and 
assignments. The discussion area concerning weekend academy classes were especially helpful, as some 
students were not sure about coming to campus for weekend academy classes but after encouragement 
and discussion with other students, decided that it would be a worthwhile venture. Students were glad to 
share their experiences with both on-campus and online classes. 

Data were collected through the use of surveys and interviews. An email request with a link to a 
SurveyMonkey survey was sent to all current IDT graduate students. The survey asked questions 
regarding the use of the online advising area. Satisfaction level, ease of use, ease of navigation, 
discussion forums, relevance, and the content of different areas (degree planning, capstone/exit options) 
were evaluated using a Likert Scale of 20 questions. Suggestions for improving the overall advising area 
and examples of how it has helped the student were asked through open ended questions. 

A Convenience Sample of six current graduate students provided a usability analysis as they 
talked aloud while they walked through the advising area. A follow up focus group of these students 
provided a deeper discussion and feedback for improvement of the site and their experience using the site. 
The reviewers looked deeper with prompt questions such as what is working, what improvements do you 
suggest, and which area is used most frequently. The department chairperson and graduate advisor were 
interviewed in person to find out if the advising site has made their job easier, what suggestions they had 
for improvement, and what, if any, feedback they had received from students. 

Preliminary findings of the project and implications for using online areas for supporting students 
will be presented at the session. We will discuss how delivery and revision of the advising area occurred 
and discuss further plans for revision of the site with a focus on online students. Issues encountered and 
potential instructional strategies will be also discussed. References and suggested guidelines will be 
provided. 
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Introduction 

The practice of open learning in recent years has provided many opportunities for higher education to generate 
revenues (Huijser, Bedford, & Bull, 2008). After years of promoting OER around the global, however. The critical 
mass of OER has yet been reached among higher education and adult education (Open Educational Quality 
Initiative, 2012). Insufficient skills to use technology to access OER content and to share OER contents have been 
the major obstacle to expand the usage of OER (D’Antoni, 2009, p.6). Many internal and external factors are 
playing parts in impeding prospective OER users’ decision to engage with the open learning processes. 

Many cognitive skills are necessary in order to fully embody the pedagogical benefits of OER. Hilton III and 
colleagues (2010) identified four “R”s that OER users should prepare to conduct in order to hilly take the advantage 
of the OER, which are (1) reuse, (2) revise, (3) remix, and (4) redistribute. In addition four contextual factors can 
further impact the level of success in the four “R”s (Hilton III et al., 2010), which are (1) access to editing tools, (2) 
level of expertise required to revise or remix, (3) meaningfully editable, and (4) source-file access (i.e, ALMS 
factors). There are also concerns on many internal factors, such as motivation, could impact the level of acceptance 
among prospective and current OER users (Huang, Lin, & Shen, 2012). Existing literature, however, has not 
provided conclusive evidences to explain the relationships among all interacting factors. 

To address this issue, this study investigated learners’ motivation systems that drive them to interact with OER 
systems by utilizing the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh, Morris, Davis, 
& Davis, 2003). The following literature survey discusses the issues facing OER systems, the importance of 
understanding learner motivation in open learning, and the efficacy of UTAUT in understanding the composition of 
open learners’ motivation systems. 


Background 

Open learning has long been praised for its positive impact on modern education system, which in recent years has 
provided many opportunities for higher education to generate revenues and publicities. Ranging from early 
trailblazers (e.g., Open Courseware at MIT) to recent iterations (e.g. Coursera at Stanford), all these initiatives have 
provided examples for organizations to share educational resources online. 

In spite of the attention focusing on OCW and MOOCs, recent literatures are concerned with the sustainability of 
OER. Open Educational Quality Initiative (2012) concluded that the critical mass of OER has yet been reached 
among higher education and adult education. Among many reasons, lack of skills to use technology to access OER 
content and more importantly, lack of skills to share OER contents have been the major hurdles in growing OER 
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(D’Antoni, 2009, p.6). While higher education institutions are eager to share their courses online for free, 
prospective learners might be reluctant to interact with the open courseware systems due to many factors external 
and internal to learners. 

Interacting with an OER system is not an easy task. Hilton III and colleagues (2010) identified four “R”s that OER 
users should prepare to conduct in order to flilly take the advantage of the OER, which are (1) reuse, (2) revise, (3) 
remix, and (4) redistribute. Furthermore, there are four contextual factors that can impact the level of success in 
carrying out four “R”s (Hilton III et ah, 2010), which are (1) access to editing tools, (2) level of expertise required to 
revise or remix, (3) meaningfully editable, and (4) source-file access (i.e, ALMS factors). Clearly the facilitating 
conditions as external factors, can influence the utilization of OER. There are also concerns that when adopting new 
technologies for open learning, many internal factors, such as motivation, could impact the level of acceptance 
among target audiences (Huang et al, 2012). Existing literature, however, has not generated satisfactory empirical 
evidences to explain the relationships among all interacting factors that ultimately can impact the level of learner 
acceptance towards OER. 

To address the aforementioned issue, this research adopted the Unified Theory for Acceptance and Use of 
Technology (Venkatesh, et ah, 2003) to investigate the motivation system among open learners upon interacting with 
a Chinese-based open courseware, MyOOPS, in Taiwan. MyOOPS, as a non-profit operation, has served Chinese- 
literate open learners around the world since early 2000. All open educational contents on MyOOPS are translated, 
edited, and organized by volunteers who are fluent in English and Mandarin Chinese. The portal has reached tens of 
thousands of open learners in the past decade. 


Method 

This online survey study targeted all MyOOPS users around the world who can understand written Traditional 
Chinese. The online survey replicated the UTAUT instrument on a 7-point Likert scale with minor modification and 
was translated into Mandarin Chinese, which consisted of seven predicting factors, with 26 items (performance 
expectancy, effort expectancy, social influence, facilitating conditions, self-efficacy, anxiety, and attitude towards 
using technology) and one outcome factor, intention to use the technology, with three items. In addition, 
participants’ demographic information was collected (i.e., gender, age, education level, and current occupation). 

Recognizing potential opportunities for participants to misinterpret the meaning of the translated survey, the research 
team adopted two procedures to mitigate such risk. First, upon converting the UTAUT instrument to English, we 
adopted the back-translation process (Brislin, 1970), which consisted of an independent translator translating the 
“translated version” from Mandarin Chinese back to English, to identify if there is any significant discrepancy 
between the original and “back-translated” versions. Second, after the back-translation process, the research team 
conducted a focus group on the translated survey with three content experts on educational research, survey 
research, and information systems. All focus group members are fluent in Mandarin Chinese and American English. 
Revisions were made on the translated survey as the result of both procedures. 

The online survey link was posted on the entry page of MyOOPS ( http://www.myoops.org/main.php ) that is 
available 24/7, 365 days a year for the entire data collection period. The survey in average took less than 15 minute 
for each participant to complete. All participation was voluntary and participants could withdraw from the study any 
time they desired. This study gathered 729 usable survey responses between July 20, 2010 and July 18, 2012 from 
4,235 online participants for data analysis. 


Data Analysis and Results 

Among 729 participants who completed the online survey, 462 of them are male (63.37%) and 263 of them are 
female (36.08%). The largest age group was between 21 - 25 year old (n=214, 29.36%) followed by the group of 26 
- 30 year old (n=128, 17.56%). In terms of education level, the majority of MyOOPS users reported to have 
obtained a bachelor’s degree as their highest academic degree (n=419, 57.48%). There were 154 participants who 
reported to have a Master’s degree as their highest academic credential (21.12%). 
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In terms of occupation, the largest group, 249 participants (34. 16%), selected “other” as their profession while the 
next largest group was the education and scholarly research group (n=123, 16.87%). Considering the age 
distribution among MyOOPS users, a recent case study suggested that the “other” occupation category might consist 
of participants who were either between jobs or were about to enter the job market for the first time (Huang, et ah, 
2012 ). 

The inferential statistical analysis in this study consisted of reliability testing, construct validation, and common 
approaches to develop measurement and structure models in the context of structural equation modeling. The 
reliability testing reported that all UTAUT constructs had Cronbach’s alpha scores greater than .70, indicating 
acceptable construct reliability. See Table 1 below. 


Table 1. Descriptive statistics and reliability measurements. 


Construct 

Mean 

Standard Deviation 

Cronbach’s Alpha 

Effort expectancy 

6.41 

1.44 

0.93 

Attitude toward using technology 

6.72 

1.41 

0.92 

Anxiety 

4.09 

1.70 

0.92 

Performance expectancy 

6.17 

1.40 

0.85 

Social influence 

5.73 

1.49 

0.90 

Facilitating conditions 

6.73 

1.41 

0.89 

Self-efficacy 

5.82 

1.29 

0.92 

Behavioral intention to use 

6.76 

1.42 

0.97 


Upon conducting EFA, parallel analysis, and reliability testing, all items and eight factors remained in the scale with 
good reliability (Cronbach’s alpha > 0.85). In order to understand the relationships among all internal and external 
factors impacting MyOOPS adoption among open learners, a structural equation modeling analysis was conducted 
with two data analysis stages. First, confirmatory factor analysis (CFA) was used to assess the validity of the 
indicator items of the measurement model. The analysis reported an adequate model fit (GFI=0.92, AGFI=0.90, 
CFI=0.98, IFI-0.98, NFI=0.99, RMR=0.042, RMSEA=0.049). 

Second, we assessed the structure model suggested by UTAUT. The GFI, AGFI, CFI, IFI, NFI values were 0.92, 
0.90, 0.98, 0.98, and 0.99, respectively. The RMR value was 0.041, and RMSEA was 0.048. These metrics indicated 
an adequate fit between the hypothesized model and the observed data. 

The analyses reported a strong predictive power (R 2 = 0.54). Among internal factors, attitude (J3= 0.44,/? < 0.001) is 
the strongest predictor of behavioral intention, followed by effort expectancy (J} = 0.20) and anxiety (J3= -0. 12), but 
self-efficacy had no effect. With respect to external factors, social influence (fi= 0.15,/? < 0.05) and facilitating 
conditions (/?= 0.22, p < 0.001) showed direct effects on behavioral intention, but performance expectancy had no 
effect. See Figure 1 below for the results of the identified model. 
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Results of standardized LISREL estimations. 


Discussion 

Our findings suggested that internal factors of leaners play a bigger role than external factors in supporting learners’ 
intention to use MyOOPS. This outcome implies that, in contrast to the design of formal online learning 
environments, it might be less important to emphasize extrinsic incentives while designing open learning 
environments. Second, the study revealed the minimal role of self-efficacy for using MyOOPS. We contended this 
was the direct result of the goal-less open learning environment, which learners’ perceived self-efficacy during the 
open and self-directed learning process became somewhat irrelevant. Both findings have design implications, to a 
certain extent, on how OER and open learning environment designers should prioritize design tasks for the most 
efficient learning processes. Future research should focus on how the openness of OER and open learning 
environments impact learners’ goal-setting process without explicit learning goals and how to engage open learners 
long-term without tangible goal structures. 
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Abstract 

The objectives of the research is to determine the impact of internet media use to facilitate learning for secondary 
school student. This research is a quantitative descritive research using an survey method. The primary data obtained 
through the quetioner and forum group discation. The research sample consists of 207 students which is taken from 
class IX at East Jakarta Secondary School (SS). This research was conducted during September to October 2012. 

The results show that: (1) More then a half (62,25%) agree, 32,27% no coment, and 5,48% all of student not agree, 
Internet media is tutor or teacher; (2) Almoust a half (38,25%) no coment, 45,6% agree, and 16,15% all of student 
not agree internet media as socializing agent. (3) Almoust a half (44,44%) agree, 33,36% no coment, and 22,2% all 
of student not agree, internet media as motivator for learning; (4) More then a half (65,16%) agree, 27,22% no 
coment, and 7,62% all of student not agree internet media as problem solving. The conclusion internet as media: (1) 
can assist students in understanding the material in the students' learning, (2) can enrich, clarify, simplify learning. 
(3) .can increase interest, making diligent in finding information related to learning, (4) can encourage motivation 
spirit, adding the initiative in learning, and (5) can help solve, reduce the difficulty and can complete tasks in the 
study. 


Preliminary 

In the current era of globalization, the development of information technology, especially the internet very rapidly in 
various countries such as Indonesia. Present Internet as a medium that integrated all existing conventional media. 
Internet access to the global world to obtain the required information, including the exchange of data and 
information. Application and development of the internet media in the world has entered pervasive education and 
learning, as well as in Indonesia. Internet in education is often used as a medium of teaching and learning. Internet 
media can facilitate and enhance student learning. Definition of educational technology set by the Association for 
Education Communications and Technology (AECT), namely "education technology is the study and ethical 
practice of facilitating learning and improving performance by creating, using, and managing processes and resource 
appropriarite technology" (Januszewski and Molende 2008, p.l. and Reiser & Dempsey, 2012. P.5). Through the use 
and management of appropriate technology such as the internet media allows every student be able to improve the 
self-learning process (individual learning) or face to face at SS. Utilizing the internet media will facilitate students in 
accessing a variety of information science, school assignments sent by email, and so on. In addition to students, 
teachers also can facilitate in implementing the learning process. Use of Internet media is essential for the attainment 
of learning on SS, considering this system prioritizes students to learn individually to support face to face learning. 

The Internet is a very large computer network that consists of small networks which interconnected and reach out 
to the whole world or the internet is often called the Interconnected Network (Deanie French, Charles H, C Johnson 
and Gerald Farr, 1999, p.25; Oetomo, 2002, p.32; Rakhmawati, 2009, p.30). The computers were stand-alone or 
independent nature but each other are connected . 

Meanwhile (Gagne, 1970) stated, "The media are various components in learners' environment which support the 
learners learning". Media has many definitions ranging from a special form of communication such as printing and 
nonprinting form, either by design or by utilitation. Media is part of the learning resources that exist outside of the 
students and be considered to influence the learning process (Percival and Ellington, 1998, h.65; Edgar Dale, 1969, 
P-50) . 
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There are three forms of learning systems through the Internet that can be considered as a basis for the development 
of a learning system by utilizing Internet, namely : (1) Web Course, (2) Web Centric Course, and (3) Web Enhanced 
Course (Haughey, 1998, h.72). 

According to R.E. Clark , there are five function of media (Internet) learning. Clark argued that "the condition under 
media can be made to influence and impact learning are being optimally Explored from at least five perspective : (a) 
media as technology, (b) media as a tutor or teacher, (c) media as Socializing agents, (d) media as motivators for 
learning, and (e) media as problem solving" (Plomp and Ely, 1996:69). 

First, media as technology .Utilization or resource is done in order to make the learning resources can be used as a 
source to obtain information in general is done. The use of the Internet media for learning purposes, in which all 
learning materials, discussion, consultation, assignments, exercises and exams delivered entirely over the Internet. 
Students are entirely separate, but relations or communications between learners can be done with the teacher at any 
time. Communication is mostly done on an ongoing basis. This course web form does not require any face-to-face 
activities, either for learning or evaluation and examination, because all the learning process is done entirely through 
the use of internet facilities like e-mail, chat rooms, bulletin boards and online conferencing. Furthermore Spector 
(2012:5) defines technology is the application of knowledge for a practical purpose. 

Secondly, media as tutor or teacher. Use of learning resources such as the Internet is based on the premise that 
learning resources be used to provide additional resources, informationand enrich the learning materials according to 
the science disciplines studied. Students can learn independently achieve material resources in order to enrich the 
learning process. The foundation for the use of media (internet) as a communication tutor education, namely, the 
deliberate and intentional act of communicating content to students with the assumption that they will learn 
something "from" these communications (Krendl, Ware, Reid, & Warren, 1996). For example, the animation is a 
form of media that can be delivered to students through a variety of technologies such as the World Wide Web. In 
web-based science, such as the movement of moon around the earth animation may be shown to the students 
through visualization and animation are supported with audio and text. 

Third, the media as socializing agent. With the utilization of instructional media is expected to influence the 
attitudes or behavior. Media can be used to change the behavior of the users (student). With the web students more 
interested to want to learn, be happy to learn and have a desire to learn. 

Fourth, the media as a motivator for learning. In this case the learning resources that can be utilized should be a 
motivator in self-leamers to learn more and develop a sense of want to explore what is learned. 

Fifth, the media as problem solving. In this case the use of media such as the internet or learning resources are 
expected to be used as a thinking tool to solve the problem. Internet usage is expected to affect the creativity of 
students who appear to facilitate the understanding especially subjects being studied. When we refer to the Clark 
opinion, the use of the internet can be a path associated with the learning process and its learning objectives to be 
achieved. 

Research Questions 

Based on theories or concepts mentioned Clark above, researchers want to conduct research under the title "Impact 
of Internet Media Use to Facilitate Learning For Secondary School Student”. 

Based on the above, the general question in this study is whether the use of the Internet as a medium has a positive 
impact on high school students to facilitate learning? Particular research question, focusing only on the 4 questions 
out of 5 questions for the media as technology. Based on assumtions nearly 90% of student regard the media as 
technology. Hence the question whether students believe that focuses internet media can serve as: 

a. tutor or teacher (media as tutor or teacher) ? 

b. socialization agents (media as socializing agent) ? 

c. learning motivator (media as a motivator for learning) ? 

d. learning problem solver (media as problem solving) ? 
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Research Methods 


Based on the research objectives, namely to get an idea of how to use the Internet as a medium or a source of 
learning, this research enter into descriptive survey research explorative. With the survey method and a descriptive 
report, the data obtained from respondents collected, compiled and then analyzed. According to Gall and Borg 
(2007:300-301), descriptive research is more focused on what the question is. Such as how many teachers or 
students who argue or take a stand on the phenomenon exists. In other words intended to collect information on the 
status of symptoms at the time the study was conducted, describe what it is about a symptom or condition. 
Descriptive researchers developed the concept and gather facts, but does not test the hypothesis. 

The population is all of the Hight School Student at 3 locations, namely : public SS 21, public SS 36, and public SS 
3 1 all of SS located in East Jakarta, while all reasonable population of Grade IX student, from tthree SS. Affordable 
sample at 207 class IX from SS the three locations. This study was conducted in September-October 2012. 

The method used was a survey method, a tool is selected as the data collection was questionnaire which have 
the form likerts scale with options 1-3, which means to agree, ordinary, and did not agree. If the respondent chose 
number 1 is defined argued agree, 2 is explain nocomment, and 3 is disagree (not agree). Questionnaire before used 
to collect the data, first tested the validity and reliability. Validity test was calculated with Product Moment formula 
and reliability test was calculated with Cronbach Alpha formula. Level of reliability of the instrument is 0.767 and 
can be categorized as good. Data were analyzed by using frequency the number of voters in each category and the 
percentage calculated. Then calculated the Standard Deviation (SD) and the mean of each group of data by each 
subvariabel in this study. The data was then analyzed by comparing the mean using the t test. 

Results 


1. Utilization of Internet Media in Learning 

Toseethe impact ofthe use ofinternetmediain learningare presentedinthetable below: 

Tablel. Utilization of Internet Media In Learning 


The internet use 

Agree % 

Not Comment 

% 

Not 

Agree % 

Mean 

Standar 

deviation 

1. Media internet is tutor or techer 

62.25 

32.27 

5.48 

2.57 

0.65 

2. Media internet as socializing agent 

38.25 

45.6 

16.15 

2.22 

1.8 

3. Media internet as motivator for 
learning 

44.44 

33.36 

22.2 

2.21 

1.04 

4. Media internet as mental tools for 
thinking and problem solving 

65.16 

27.22 

7.62 

2.58 

0.75 


Based on the table above, the Internet as a medium for mental as for thinking and problem solving is higher than 
others, it is seen from the answer agrees 65.16, 27.22 and not coment agree not 7.62. This proves that the internet 
media use, students can complete all the issues relating to their learning tasks given by the teacher. While most of 
student, internet media as a motivator for learning, seen from respondents stating agree 44.44%, 33.36% and not 
coment not agree 22.22%. It is very reasonable because after all the learning needs of students are met on the 
internet then there will be motivation in itself to learn. 
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1. Media 
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techer 


2. Media 
internet as 
socializing 
agent 


3. Media 
internet as 
motivator for 
learning 


4. Media 
internet as 
problem 
solving 


Graph 1. Utilization of Internet Media By Students 


□ Agree % 

■ Not Comment % 

■ Not Agree % 


2. Internet media is tutor or teacher 

Based on the research results explain that Internet media is tutor or techer. Based on respondents who were 
interviewed stated richer knowledge with internet usage is higher than others, it is seen from the answers 
respondents stating agree 70.45%, 27.01% and not comment not agree 2.68%. With the Internet, all required 
information is available, including all the materials that have not been explained by the teacher already provided on 
the internet with various writings, so that the open Internet will enrich the students' knowledge on all subject matter 
related to the learning needs of students. While the lowest is the plumpness 53.05% respondents agree, not coment 
39.3% and 7.65% agree not understanding the internet if you use the internet more and more, because of all course 
materials are provided on the internet if it is not used then the students do not get knowledge even if you just rely on 
explanations from teachers in understanding the subject matter. Of this data can say that the Internet can enrich, 
clarify, simplify learning. To clarify the above data is presented in graphical form below. 



enriching the richer clarify the all difficulties deepen deeper 

knowledge knowledge subject matter can be in clear knowledge understanding 
about the is difficult of the material 

subject matter 


■ Agree % 

■ Not Comment % 

■ Not Agree% 


Graph 2.InternetMedia as a tutor or teacher 
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3. Internet Media as Socializingagent 

Based on the results of research conducted, it can be explained that as the Internet Media Socializing agent, based 
on the answers of those interviewed stated that there is homework, it will search the internet higher than others , it 
is seen from the answers respondents stating agree 6, 89%, not comment not agree 19.56% and 2.75%. Use of the 
Internet to complete school assignments in the subject matter students are available on the internet. While the 
lowest will improve student learning, it is seen from the answers respondents stating agree 29.1%, not comment 
46.69% and 24.21% not agree.. Not comment 9.55% , and 2.55% not agree. Will improve student learning are 
formed when students are able to understand precisely the internet in search of instructional materials on the 
internet for the needs of school students' learning task. Of this data can be said that the Internet can increase 
interest, making diligent in finding information related to learning. To clarify the above data is presented in 
graphical form below. 



finding the the opiate more homework open the increased 
information open studious can be Internet at interest in 
yourself internet completed school and learning 

at home 


■ Agree % 

■ Not Comment % 

■ Not Agree% 


Graph 4. Internet Media as socializing agent 
4. Internet media as a motivator for learning 

Based on the results of research conducted , it can be explained that the internet media as a motivator for learning 
internet that states can improve efforts to achieve maximum learning outcomes is higher compared with other 
statements, it is evident from the answers respondents stating agree 65.7%, not comment 19.1%, and 15.2% not 
agree. Maximum learning is achieved when students actually use the Internet as a learning resource. While the 
lowest of the answers respondents stating internet help completing school assignments, it is seen from respondents 
who stated that 29.3% agree, not comment 37.94% and 32.76% not agree. School work will be completed when 
students used the internet as a source of learning proper, appropriate use and how to find the right material is also 
the subject matter. Of this data can say that the Internet can encourage the spirit, adding the initiative in learning. To 
clarify the above data is presented below the graph. 
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in learning learning the spirit of learn high in initiative habit of 
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Not Agree% 


Graph 5. Internet Media as motivator for learning 
5. Internet media as problem solving 

Based on the results of research conducted, it can be explained that the internet media as mental tools for 
thinking and problem solving in accordance with the results of interviews with the respondents said the Internet 
helps in completing the task of the school is higher than the other statement, it is seen from respondents who 
stated 72.36% agree, not agree comment 17.645 and not 10%. The Internet provides a variety of materials to be 
used as a source in completing school assignments given to the student teacher. While the lowest quickly 
understand course material while using the internet, it can be seen from the answers respondents stating agree 
47.13%, 43.31% and not comment not agree 9.56%. This happens because the internet media is used as a 
source of a second after the explanation of learning from teachers and books, when open is not available in 
solving the problems the students use the Internet to complete their learning problems. Of this data can say that 
the Internet can help solve, reduce the difficulty and can complete tasks in the study. To clarify the above data 
is presented below the graph. 



help 

complete 
the task 


complete 
the task on 
time 


learning 
difficulties 
can be 
resolved 


more quickly 
understand 
the material 


help solve 
learning 
difficulties 
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difficulty of 
learning 


■ Agree % 

■ Not Comment % 

■ Not Agree% 


Graph 6. Internet media as mental tools for thinking and problem solving 

6. Comparison of Utilization Impact of Internet Media in Learning 
Table 7. Comparative Impact of Media Use of Internet in Learning 
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The internet use 

Mean 

Standar 

deviation 

1. Media internet is tutor or techer 

2.57 

0.65 

2. Media internet as socializing agent 

2.22 

1.8 

3. Media internet as motivator for learning 

2.21 

1.04 

4. Media internet as mental tools for thinking 
and problem solving 

2.58 

0.75 


Based on the table above it can be concluded that there are differences in the average utilization of the Internet as a 
learning medium. Internet media as mental tools for thinking and problem solving have averaged higher than the 
other is 2.58. Internet store all materials required course materials by students to use as a whole. In addition to the 
facilities available and the internet very easily obtained. While the average low is that the internet as a motivator for 
learning is 2.21. It is very reasonable because the motivation of the use of the Internet will arise when students are 
used to and able to complete school assignments given to her teacher then there will be motivation to seek answers 
to these questions on the internet.... 


Discussion 

1 . Internet media as tutor or teacher 

Internet as a tutor or teacher that the internet can enrich knowledge about any subject matter, the internet can clarify 
the subject matter which felt difficult when learningon their own, learning difficulties at school can be clarified 
through the internet, the internet can deepen knowledge of the subject matter, understanding increased if using 
internet. Internet characteristics among others are: 1). Utilizing electronic technology services; whereby teachers and 
students, students and fellow students or teachers and fellow teachers can communicate with relative ease without 
being limited by the things that precedence; 2). Uses instructional materials shall be independent (self-learning 
materials) are stored in the computer so that it can be accessed by teachers and students anytime and anywhere and 
who needs it and 3). Utilizing learning schedule, curriculum, learning outcomes and progress matters relating to the 
administration of education can be viewed at any time on the computer. This is consistent with what is conveyed by 
Miarso (2004:40), Utilization of the Internet through e-learning can not be separated from internet services. Because 
learning techniques available on the internet so complete, then this will affect the task of the teacher in the learning 
process. Previously, the teaching-learning process is dominated by the role of the teacher, because it is called the era 
of teacher. Now, the teaching-learning process, are dominated by the role of teachers and books (the era of teacher 
and book) and on the future of teaching and learning process will be dominated by teachers, books, and technology 
(the era of teacher, books, and technology). Another theory says the use of the Internet as a learning resource 
belongs to the individual learning system. ways that are considered easy and affordable by learners. Delivery of 
teaching materials can also be accompanied by a tutorial program, which was organized by a particular location or 
schedule and in accordance with the collective agreement. 

2. Internet media as socializing agents 

Internet media as socializing agents to seek information from the friend asked, the more opiate open internet for 
learning, more studious once accustomed to using the Internet, every homework can be found on the internet, the use 
of the Internet at school and at home can increase interest in learning. Internet learning by utilizing website as a 
learning resource does not focus on the teacher as the teacher. This is because the focus of learning preferred on 
students’ independently. It is based on the theory according Wedermeyer (1973:73) as cited by Keegan (1996:59), 
defines self-learning as follows; Independent learning is that changed behavior, that results from activities carried on 
by the learner in space and time. Whose environment is different from that of the school, learners who may be 
guided by teachers, but who are not dependent upon them, learners who accept degrees of freedom and 
responsibility in initialing and carrying out the activities that lead to learning. Of this quotation is called self- 
learning is changing the learning behavior results from activity conducted by the students in time, place, and 
different learning environments to schools, students are guided by the teacher but does not depend (entirely) to them, 
students gain the freedom and responsibility in organizing and carrying out learning activities. In independent study, 
students have the freedom to learn without having to attend lessons the teacher. Students can learn the subject or 
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topic of a particular subject by reading the print module, see and listen to books or instructional media program 
without assistance or with limited assistance from others. 

3. Internet media as motivator for learning 

Internet media as motivator for learning the Internet can add to the attractiveness of learning, the Internet can 
improveefforts to achieve maximum learning outcomes, the Internet can encourage the spirit of learning, adding 
internet learning initiative, the Internet can reduce the habit of asking to friend. 

Change of behavioris the result of the learning process as result of deliberate inter action between students and their 
learning environments. Threefactorsthat encouragestudentsandteachersteaching and learning process, namely: 1). 
Physicalandmentalreadinesstodo something, 2). Self-motivation isthe urgetodo something, and3). Objectives to be 
achieved. 

4. Internet media as problem solving 

Internet mediaasmentaltools forthinkingandproblem solvingto helpcomplete the taskof theschool, the Internetcan 
completetasksin a timely manner, alllearning difficultiescan beresolvedby goingon the Internet, the 
Internetcanhelpunderstand the subject matter, the lnternethelpssolvelearning difficulties. This is consistentwith the 
theory of Batesl995 and Wulf, 1996 as quoted by Sudirman Siahaan, namely: 

a. Increasing levels of learning interaction between learners and teachers or instructors (Enhance interactivity). 

b. Allows the interactionof leamingwhereand at any time(time and placeflexibility). 

c. Reachlearners incoverage(potentialtoreachaglobalaudience) 

d. Facilitate the completion and storage of learning materials 
(easy updating of content as well as archivablecap abilities). 

Conclusion 

The impact of the use of the internet media in learning are: 

1 . Internet media as a tutor or teacher that the internet can enrich, clarify, simplify learning. 

2. Internet media as socializing agent that the internet can increase interest, making diligent in finding 
information related to learning. 

3. Internet media as motivator that the internet can encourage motivation spirit, adding the initiative in learning. 

4. Internet media as learning problem solving that the internet can help solve, reduce the difficulty and can 
complete tasks in the study. 
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Abstract 

The purpose of this study is to examine the effects of multimedia e-book on elementary students’ English 
achievement. Two hundred fourteen sixth grade students were assigned into one of the three groups: learner-directed 
multimedia e-book group, instructor-directed multimedia e-book group, and face-to-face group that used traditional 
textbooks. A growth model of multi-level analysis was conducted to investigate the relationship between multimedia 
e-book and English achievement. More analysis was conducted to examine the change of students’ academic 
achievement for two months. The results were summarized as follows: multimedia e-book was effective for students 
rather than the conventional text-book in learning English. Learner-directed multimedia e-book group outperformed 
than instructor-directed multimedia e-book group in English knowledge gains. 

Research Background and Research Questions 

Focusing on the significant learning effects of multimedia in language education, education ministry of 
Korea developed multimedia e-book as part of the innovative English education in Korea. However, validity of 
multimedia e-book should be examined prior to implication. Moreover, most empirical educational research has 
examined whether multimedia is merely used or not rather than how multimedia should be implemented and applied 
in classroom settings. Therefore, this research focuses on obtaining reliability of multimedia e-book in terms of 
learning effects and investigating the optimum use of the multimedia e-book in class. In relation to the use of 
multimedia e-book, who would be more central subject between instructors and students in classrooms is an 
important issue to be recognized. Students should be self-directed learners to use the latest technology such as 
multimedia e-book, since multimedia technology allows autonomous exploration, information on demand, and 
authentic learning (Murray, 1999; Zumbach, Kumpf, & Koch; 2004). On the other hand, the instructors should have 
a pivotal role that could integrate the whole process of online classes, and should be a coordinator among students 
and computers in multimedia online courses (Brown, 2007). However, few studies have examined that who should 
have predominance actually in multimedia web-based learning. 

The purpose of this study is to examine the effects of multimedia e-book compared to conventional English 
instruction and compare the effectiveness between learner-directed multimedia e-book learning and instructor- 
directed multimedia e-book learning. Research questions were set as follows: 

(1) Is there a significant effect of multimedia e-book in English achievement? 

(2) Is there a significant difference between learner-directed multimedia e-book learning and instructor-directed 
multimedia e-book learning? 


E-book and Multimedia E-book 

Along with advancement of technology all kinds of “electronic” information can be accessed online or 
through the reader devices, thus digital publishing have been significant in teaching and learning. Electronic books 
(e-books) are becoming affordable due to easy accessibility and economic efficiency in size, weight, and purchase 
price (Pappas, 2009). Currently, most students are the technically savvy to use e-books and instructors are willing to 
utilize them as auxiliary materials for effective teaching. 

The need for e-books in the educational field is steadily being improved due to lots of their advantages: (1) 
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multimedia information, (2) flexibility, and (3) economic efficiency. First of all, e-books provide multimedia 
information including hypermedia information. While printed books consist of the majority of texts and sparse 
illustrations or tables, e-books are easy to include multimedia functions such as audio, video, and animation that 
enhance students’ interests. Flexibility in searching and browsing is another crucial point when we mention the 
advantages of e-books. They not only allow to search full-text and to link hypermedia information even with low 
hand dexterity, but also are facile to highlight, annotate, underline and bookmark. Economic efficiency of e-books 
offers a possibility that e-books can be a substitute for printed books. They are free to be utilized and customized by 
lots of students and teachers at the same time, but at remote places (Anuradha & Usha, 2006; de Jong & Bus, 2002). 

Although e-books are associated with multimedia information as mentioned above, e-books are needed to 
be differentiated against multimedia e-books. While e-books are generally known as ‘books that are available in 
digital format’ (Anuradha & Usha, 2006, p. 56), multimedia e-books not only contain integration of multimedia 
features such as texts, graphics, sounds, animations, and videos to deliver information, but also focus on web-based 
interaction between contents and users. By incorporating a variety of multimedia functions into e-books, multimedia 
e-books provide more authentic learning environments especially in language education. 

In terms of multimedia e-books, lots of studies have reported that they contribute to improvement of 
language skills such as vocabulary and listening and reading comprehension (e.g., Korat & Shamir, 2008; Shamir, 
2009; Segers, et al., 2004). A more significant fact of multimedia e-books is that students can learn language by 
themselves without instructor guide or instruction since they can utilize narrator’s reading or words guided by the 
multimedia features of e-books (Segal-Drori et al., 2010). On the other hand, there have been several studies that 
instructor’s intervention is required to help students accomplish the tasks and support cognitive and emotional 
aspects and technical issues (Nir-Gal & Klein, 2004; Tzuriel & Shamir, 2002). Thus, students’ independent 
multimedia e-book study and teacher’s instruction could be a considerable issue to utilize it and to investigate its 
effectiveness. 


Method 


Participants 

Participants were 214 sixth grade students in English class from an elementary school in Korea. A total of 
62 students were assigned to instructor-directed multimedia e-book group, 61 students were allocated to learner- 
directed multimedia e-book group, and 91 students were distributed to normal English class group. 

English Achievement 

Students’ English achievement was measured by National Assessment of Educational Achievement 
(NAEA) from Korea Institute for Curriculum and Evaluation (KICE) by using equivalent tests from three sequential 
years. Score charts for transforming raw scores to scaled score were also provided by KICE. The question items of 
the NAEA consisted of multiple-choice fn — 24) and short-answer questions (n — 6 ). The total of raw score in English 
was 60, but scale score was used in this study reported by using the method of equating. Each of reliabilities for two 
pre-tests and one post-test in our data was .83, .85, and .86. 

Multimedia e-book 

Multimedia e-book was developed by the Korean Ministry of Education and Human Resources 
Development in 2006. Multimedia e-book included a variety of learning activities such as games, flash animations, 
video, and fairy-tale animations. In addition, multimedia e-book was designed for improving communication skills, 
particularly in areas of learning to speak. Interactive voice recognition technology was used to facilitate the learning 
activities. Students could write their own composition, and revise their writing or composition errors with the help of 
natural language process (NLP) technology embedded in the multimedia e-book. Multimedia e-book was made of 
sixteen chapters and four story telling. Each chapter was consisted of listening, speaking, reading, writing and 
supplementary activities. Story telling was not included in the study because it needed extra class hours. 

Procedures 

Students met twice a week for 40 minutes in English class for eight weeks. One chapter consisted of four 
learning modules and one chapter per two weeks was finished. One English teacher who partnered with the research 
team managed a total of sixteen learning modules for three groups. The experimental group utilized multimedia e- 
book and the control group were taught with English textbooks. While the students in learner-directed multimedia e- 
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book group studied individually using their computers, the instructor supervised them in order to concentrate on 
their learning materials. The students in instructor-directed multimedia e-book group studied together looking at a 
big screen displaying e-book manipulated and guided by the instructor. 

Data Collection and Analysis 

The first pre-test was completed. After four weeks the second pre-test was measured to estimate the natural 
growth of three groups. During the four weeks all of students were instructed in normal English class. After the 
second pre-test, multimedia e-book was presented to two experimental groups (learner-directed and instructor- 
directed multimedia e-book groups) for four weeks. A post-test was given to students to examine the effects of 
multimedia e-book on English achievement. Thus, students’ English achievement was measured three times. For 
statistical analysis, academically available software HLM and SPSS statistical package were used. In order to 
compare the difference between multimedia e-book group and normal class group, the data from two experimental 
groups were aggregated. 

Results 

The ANOVAs revealed that there were no significant differences in the pre-test 1 scores of each group. 
Hence, it can be concluded that the initial ability levels of the groups remained comparable despite attrition. 

A summary of the results of the HLM analyses for all of students is in Table 4. The mean estimated post-test scores 
of all participants was 167.07. The mean growth rate was 2.68 scaled score points per term. There was significant 
variability among students in term of their status at post-test (x 2 =1589.36, p<.001) and growth rate (% =467.93, 
p<.001). In other words, all participants’ academic achievement increased significantly from the first pre-test to 
post-test. 

Table 1 . Learning effects for all participants 


Fixed effect 

Coefficient 

Standard 

Error 

t-ratio 

df 

Post-test score (n 0 ,j) 

Mean post-test of all ((3 0 o) 

167.07 

0.64 

262.54*** 

213 

Growth rate (jti,) 

Mean growth rate of all ((3 1 0 ) 

2.68 

0.79 

3.40** 

213 

Boosts growth rate (7t 2; ) 

Mean acceleration growth rate of all ((3 20 ) 

0.17 

0.29 

0.58 

639 

Random effect 

Standard 

Deviation 

Variance 

Component 

df 

Chi-square 

Individual status at post-test (r 0i ) 

8.10 

65.62 

213 

1589.36*** 

Individual growth rate (r u ) 

2.70 

7.29 

213 

467.93*** 

Temporal variation (£„•) 

3.49 

12.19 




**p<.01, ***p<.001 

Figure 1 graphically illustrates the estimated growth curves of all participants. There was gradual growth of 
all participants from the first pre-test to post-test. 
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Figure 1. Change of learning effects for all participants 


While the mean estimated post-test scores of traditional textbook group students was 167.14, the mean 
estimated post-test scores of e-book group students was 167.02. There was no statistically significantly difference 
between the two groups. However, the difference of growth rate between e-book and traditional textbook groups 
were 3.27, and it was a statistically significant difference. There was significant variability among students in term 
of their status at post-test (x 2 =1615.69, p<.001) and growth rate (X~47 1.13, p<.001). 


Table 2. Learning effects of multimedia e-book and traditional textbook 


Fixed effect 

Coefficient 

Standard 

Error 

t-ratio 

df 

Post-test score (Jto/>) 





Mean post-test of textbook group ((3 00 ) 

167.14 

0.99 

168.68*** 

212 

Differences of post-test score between e-book and 
textbook groups ((3 0 i) 

-0.12 

1.29 

-0.09 

212 

Growth rate (jti,-) 





Mean growth rate of textbook group ((3i 0 ) 

0.80 

1.25 

0.64 

212 

Differences of growth rate between e-book and 
textbook (Pn) 

3.27 

1.60 

2.05* 

212 

Boosts growth rate (7t 2i ) 





Mean boosts growth rate of textbook group (P 2 o) 

-0.55 

0.47 

-1.17 

636 

Differences of boosts growth rate between e-book 
and textbook (P 21 ) 

1.26 

0.59 

2.12* 

636 

Random effect 

Standard 

Deviation 

Variance 

Component 

df 

Chi-square 

individual status at post-test ( r 0i ) 

8.13 

66.08 

212 

1615.69*** 

individual growth rate (r u ) 

2.71 

7.32 

212 

471.13*** 

Temporal variation ( e„j 

3.46 

11.98 




*p< 05, ***p<.001 
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Figure 2 provides a comparison of e-book group and textbook group. Figure 2 shows the achievement 
change between the e-book group and traditional textbook group was parallel from the first pre-test to the second 
pre-test, but there was a steep achievement change in experimental group during the period of using multimedia e- 
book. 



Figure 2. Change of learning effects for e-book group and textbook group students 

The mean estimated post-test scores of learner-directed e-book learning group students was 167.72, the 
mean estimated post-test scores of instructor-directed e-book learning group students was 166.34. This difference 
between the two groups was statistically significant, in addition, the difference of growth rate between learner- 
directed and instructor-directed e-book learning groups were -1.20, and it was also a statistically significant 
difference. There was significant variability among students in term of their status at post-test (x z =1025.42, p<.001) 
and growth rate (x 2 =290.00, p<.001). 

Tabl e 3. Learning effects of learner-directed group and instructor-directed group 

Fixed effect Coefficient Standard t-ratio df 

Error 

Post-test score (jco/>) 


Mean post-test of learner-directed group (Poo) 167.72 1.13 

Differences of post-test score between learner- 
directed and instructor-directed groups ((3 01 ) 

Growth rate (k i ,) 


148.23*** 121 
-0.84* 121 


Mean growth rate of learner-directed group (Pi 0 ) 

Differences of growth rate between learner- 
directed and instructor-directed groups (Pn) 

Boosts growth rate (7t 2i ) 

Mean boosts growth rate of learner-directed group 
(P20) 

Differences of boosts growth rate between learner- 
directed and instructor-directed groups (P 2[ ) 

Random effect 


4.67 

1.42 

3.28** 

121 

-1.20 

1.98 

-0.61* 

121 

0.82 

0.52 

1.57 

363 

-0.23 

0.72 

-0.32 

363 

Standard 

Deviation 

Variance 

Component 

df 

Chi- 


Individual status at post-test (r 0i ) 8.17 66.83 121 1025.42*** 
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Individual growth rate (r u ) 

2.74 

7.49 

121 

290.00*** 

Temporal variation (e,„j 

3.28 

10.73 



*p<.05, **p<.01, ***p<.001 


Figure 3 illustrates a comparison of leaner-directed e-book group and instructor-directed e-book group. 
Learner-directed e-book learning group students outperformed than instructor-directed e-book learning group 
students in both the second pre-test and the post-test. 



Figure 3. Change of learning effects for learner-directed and instructor-directed group students 


Conclusion and Discussion 

The results were summarized as follows: multimedia e-book was effective for students rather than the 
conventional text-book in learning English. Learner-directed multimedia e-book group outperformed than instructor- 
directed multimedia e-book group in English knowledge gains. In terms of multimedia e-books, these results are 
supported by lots of studies reported that they contribute to improvement of language skills such as vocabulary and 
listening and reading comprehension (e.g., Korat& Shamir, 2008; Shamir, 2009; Segers, et al., 2004). 

A more significant fact of multimedia e-books is that students can learn language by themselves without 
instructor guide or instruction since they can utilize narrator’s reading or words guided by the multimedia features of 
e-books (Segal-Drori et al., 2010). Although instructor’s intervention is required to help students accomplish the 
tasks and support cognitive and emotional aspects and technical issues (Tzuriel& Shamir, 2002), students’ 
independence is likely to be more considerable in knowledge gains. 

So far, e-book usually has been used with textbooks. However, multimedia e-book was used as a main 
learning tool in this study, not as supplementary material in regular English classes in elementary schools. By using 
multimedia e-book exclusively, teachers and educators could expect better achievement of learners on public 
education. Nevertheless, when multimedia e-book is utilized in the instructional field, researchers need 
consideration of effective instructional strategies or scaffoldings to support students’ learner-centered e-book 
learning in further research. 
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Abstract 

In this conceptual paper, we seek ways to integrate design-based research into online higher education contexts. Our 
goal is to promote participatory, formative, and strategic approaches to evaluation and research methods in online 
education. In this paper, we will review the history and basic principles of design-based research, discuss its benefits 
and limitations, and then propose a new research framework for investigating positive cases of the effective design 
of online learning environments. 

The rapid growth of online education necessitates extensive institutional research on the quality of teaching 
practices in online settings. Typically, higher education institutions in the United States administer summative 
course evaluations at the end of academic courses to inform teachers about learning outcomes and overall student 
perceptions of their learning experience. These summative assessments can be valuable to guide the institution in 
making future administrative decisions and to guide the teacher in making future instructional decisions; and such 
measures are useful to get an overall picture of students’ experiences in online settings. However, they can be 
inadequate to frilly understand the complex variables constituting the learning ecology, resulting in narrowly 
constructed conceptualizations of effective online teaching practices. Another limitation of summative measures is 
the lack of immediacy. To what extent are findings reflected back on real-life settings in a timely manner, helping 
education professionals solve related issues and problems within the learning environment efficiently? We argue 
that this after-the-fact feedback is often not received in time to make responsive changes that could potentially 
benefit the learning experience in a formative way more immediately. We also contend that an exclusive focus on 
problematic issues, while well intentioned to improve them and seek solutions, is also too narrowly focused and 
results in missed opportunity to give recognition to what is working well in order to learn from that also. 

Design-based research (Brown, 1992) is a methodological approach that has emerged within the field of 
learning sciences and is a more formative evaluative approach that can help teachers improve their online teaching 
practices and the quality of the learning experience simultaneously through ongoing cycles of design. It is important 
to note that this approach positions technology as a process rather than a mere artifact to be studied as it affects 
education and impacts learning (Amiel & Reeves, 2008). This process ideally involves all stakeholders in the 
research process, including administrators, teachers, and instructional designers, to form a research team to analyze 
design challenges, develop and implement solutions, and then continue with analysis all over again in a cyclical 
pattern of multiple iterations. In a formative and proactive way, Amiel and Reeves argue that design-based research 
allows researchers to actively engage with practitioners “in order to help direct technological development rather 
than react to it” (p. 32). 

While design-based research typically focuses on identifying design challenges and then seeking potential 
solutions to such problems, we propose that a unique approach to design-based research focusing on positive cases 
of effective design could be equally valuable. Administrators, policymakers, higher education faculty and staff, and 
instructional designers could potentially benefit from the following overview of this unique approach to DBR as a 
means to examine positive cases and innovations in designs, tools, and techniques. The institutional impact of such 
an approach also lies in its emphasis on organizational support in order to build shared knowledge. 
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Defining design-based research 


The Design-Based Research Collective (2003) defines design-based research (DBR) as "the study of 
learning in context through the systematic design and study of instructional strategies and tools," which might 
involve, but is not limited to, shared artifacts, activity structures, scaffolds, and curricula (pp. 5-6). The research 
process is iterative and is tied to everyday practices. Reeves (2006) suggests four DBR steps: (1) analysis, (2) 
development of solutions, (3) iterative cycles of testing and refining, and (4) reflection. Sometimes, however, the 
issue at hand may not require a solution, but rather, careful elaboration. For example, we might investigate when, 
how, and more importantly, why a particular instructional strategy works well, using formative qualitative 
assessment and quantitative measures (Cobb, Confrey, & diSessa, 2003; Design-Based Research Collective, 2003). 

This recursive nature of the DBR process through cycles of design, development, implementation, and 
analysis requires close collaboration between teachers and researchers in authentic settings (Wang & Hannafin, 
2005). As a result, teachers are informed in a timely manner of the instructional strategies that are working best 
because they are part of the research team. This approach allows teachers to illuminate strengths of the design and 
improve on weaker areas while actually in the process of teaching, rather than after the fact. In addition, this 
practical focus of DBR leads the research endeavor to be process oriented. For example, if the instructional method 
being used reflects a social constructivist learning paradigm, we may want to know more about the process by which 
students interact with their peers and teachers over time as they refine their understandings, challenge their 
assumptions and beliefs, and reflect upon their learning experience. Thus, such reflective and collaborative 
processes involved in DBR enable the emergence of new and perhaps more creative pathways for learning (Cobb, 
Confrey, & diSessa, 2003). Additionally, DBR strategies are used to generate usable knowledge for the wider 
educational community and could thus be applied and tested in other educational contexts or broader organizational 
structures. 


Challenges 

There are, no doubt, challenges in conducting DBR in higher education (for an in-depth discussion see 
Dede, 2005 and Flerrington, 2012). For example, the time investment involved in DBR is considerable because as a 
result of its iterative nature, the researcher needs to be closely involved in the learning environment for long time 
periods. It is also essential that teachers are open to sharing, analyzing, testing, and refining ideas in collaboration 
with others. This may leave some teachers feeling vulnerable and exposed to criticism. Ensuring they understand 
that the focus will be on what is working well may draw them in intrinsically and increase their willingness to share 
both the successes and struggles in their teaching practice. 

However, it can be argued that each of these challenges are relatively small compared to the significant 
benefits of DBR, including improving online courses and addressing the needs of learners from diverse 
backgrounds. The growing research base on DBR in the learning sciences suggests that this approach is particularly 
useful in promoting authentic learning in higher education (e.g., Pais Marden, Herrington, & Herrington, 2009; 
Parker, 2011). We are more concerned about traditional approaches to DBR being so heavily focused on problems 
and challenges, with less regard for what is working well. In seeking ways to equally validate and illuminate 
effective tools and techniques for online teaching and learning, we propose a positive case model for DBR. 

A positive case model for DBR 

In Figure 1 we illustrate a new institutional vision for conducting DBR studies in collaboration with 
teachers in online higher education contexts. In our vision, teachers and researchers who have expertise in 
educational technology (most preferably as permanent staff members) work together with a common goal to 
improve online teaching practices and seek evidence -based design principles that could be disseminated to other 
professionals working in similar areas. Such collaboration is essential in developing a shared vision, which frames 
the initial research questions and the methods to investigate the learning process as well as the learning environment. 
Inspired by the positive psychology movement, we focus on cases that lead to positive teaching and learning 
experiences and elaborate on their characteristics and the variables that influence them. This unique approach to 
DBR, we believe, could be more easily embraced and adopted by teachers and, thus, lead to a shared knowledge 
base on effective online teaching practices. It could potentially decrease understandable reluctance or resistance 
from teachers to be involved in such intimate examination of their teaching practices which might otherwise make 
them feel vulnerable or criticized. 
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The model begins with a recognition of the necessity for institutional support involving all major 
stakeholders. It also requires the investment and involvement of teachers, instructional designers, and even 
potentially administrators to serve as a research team with the task of collaborating and contributing in different 
ways depending upon their vantage point and equally valued expertise. Building upon Reeves’ (2006) iterative 
DBR model, the first step in the model for design-based research that we are proposing is to analyze the design of 
the online course or online learning environment from many different aspects, including the virtual learning space, 
the instructional methods and strategies being employed, and the technological tools being integrated in the online 
setting. In doing so, the second step of the research process is to use this data to identify positive cases of effective 
design which can be explored and examined further for variables impacting the learning experience and the 
dynamics influencing its effectiveness. The third step in the research process involves further investigation using 
formative qualitative assessment and quantitative measures in an effort to be able to elaborate on when, how, and 
more importantly, why particular learning space conditions, instructional strategies, or technological tools are 
effective or are working well in a particular setting or context. This leads to step four and the ability to implement 
and refine prototypes for design based on positive indicators and the enhancement of current teaching practices 
deemed to positively impact the learning experience. In the fifth step, further reflection and collaboration is 
necessary as the research team convenes for additional data analysis to determine evidence-based practices drawn 
from research findings that can be disseminated with others who may benefit in the organization. This fifth step in 
the research process is by no means a final step, but rather, the process continues with additional cycles. This 
approach aligns with traditional design-based research that continues to explore new angles and aspects, additional 
variables and dynamics of the online learning environment under study. 


Institutional support 

Researcher, instructional designer, teacher collaboration 


I . Analysis of the instructional methods, 
learning space, technology tools, etc. 


2 . Identification of positive cases 


5 . Reflection 


Evidence-based design 


principles 


3 . Elaboration 


4 . Enhancement of current approaches/practices, 
implementing and refining new prototypes 


Organizational dissemination 


Figure 1. An institutional framework for conducting design-based research studies in collaboration with 
teachers. We build on the Reeves’ (2006) iterative DBR cycle by focusing on the strengths of the learning 
environment rather than problematic issues. 
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Discussion 


In this paper, we have proposed a remarkably different model for design-based research focused on positive 
cases with the aim to initiate new lines of thinking and dialogue about process-oriented ways to collectively study 
effective online teaching and learning in higher education. Further, we seek empirically grounded and theoretically 
based institutional strategies to build and to disseminate shared knowledge about design solutions and teaching 
practices. To date, the proposed model has only been abstractly conceptualized thus far, and we have immediate 
plans to conduct pilot studies to validate its usefulness for higher education and to explore it in more depth. At this 
juncture, we propose this new model for DBR to our colleagues in the field in order to begin to raise important 
questions about creative means to implement situated, collaborative research in online higher education contexts 
while illuminating good practice and good design in positive, productive, and meaningful ways. Can teachers and 
instructional designers work collaboratively to find new ways of online communication that lead best practices in 
online education? Flow can higher education institutions seek feedback on the quality of online courses in a more 
timely way? Is it possible to create an interdisciplinary task force promoting situated approaches— such as DBR— to 
research and evaluation in online higher education settings? It is our sincere hope that generative discussions 
addressing these questions will spark a lively educational debate leading to bottom-up innovations, strategic design, 
sustainable change, and shared knowledge. 
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In this study we explored the relationship among personality traits (based on the Big Five Model), game 
performance, learning gains, and attitudes of forty-four students who played an educational simulation game online. 
Half of them played in physically separated dyads following a collaborative script, while the other half played 
individually. The study also used Analysis of Patterns in Time (APT) to analyze the differences among the highest 
and lowest performers in each setting. 


Theoretical Framework 

As the demand for online courses continues to proliferate (Allen & Seaman, 2012), it is necessary to 
investigate instructional strategies that may promote motivation, engagement, and learning among online students. 
Two instructional strategies that are known to be effective for promoting engagement and motivation in the 
traditional setting are cooperative learning (Johnson & Johnson, 1991; Johnson, Johnson, & Smith, 1998; Slavin, 
1995) and game -based learning (Garris, Ahlers,& Driskell, 2002; Kirriemuir & McFarlane, 2004; Prensky, 2001). 
This study explored the effectiveness of the combination of both of these strategies in the online environment. 

According to psychology theorists, personality traits, such as the willingness to cooperate and socialize 
with others, have the potential to influence the quality and level of effectiveness of the collaboration process 
(Furhnan, 1996; Kichuk & Wiesner, 1997). In general, teams composed of people who are more talkative, tolerant, 
sociable, cooperative, and gregarious tend to perform better field tasks than teams composed of introverted people 
(Bell, 2007); however, the relationship between personality traits and task performance varies depending on the 
nature of the task (Driskell, Hogan, & Salas, 1987). 

Multiple studies across several disciplines and academic levels have demonstrated that to facilitate 
cognitive benefits among all team members, it is critical to include certain elements within the collaboration process, 
such as individual accountability, positive interdependence, social skills, and group processing (Brush, 1997; 
Johnson, Johnson & Smith, 1998, Slavin, 1995). 

The field of computer-supported collaborative learning (CSCL) combines elements of the collaborative 
learning theory with the computer-mediated communication theory (CMC) (Kirschner, 2002). CSCL has as a goal to 
study how people can learn together with the help of computers (Stahl, Koschmann, & Suthers, 2006). Even though 
the CSCL field emerged since the 19909’s, studies have now proliferated with the growth of online courses. 
However, the vast majority of CSCL studies have involved asynchronous communication tools (e.g., discussion 
forums) or text-based synchronous communication (e.g., chat tools). In this study, participants playing in dyads 
communicated with each other verbally through voice over IP. 

Purpose of the Study 

The present study was aimed at exploring whether some personality traits were related with performance 
when playing collaboratively an online educational game. 

Specifically, the study sought to answer the following research questions: 

1) What is the relationship between personality traits and performance, learning and attitude in students 

playing an online instructional game either individually or collaboratively with a peer? 

2) Are there any common patterns in the game play strategies used by higher performing students within 

each setting (individual or collaborative)? 
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Methods 


Participants 

To address these questions, 44 students from a Midwestern University were recruited using the following 
criteria: (a) be in an undergraduate or graduate academic program, (b) must have not previously played any version 
of the Diffusion Simulation Game (DSG), which was the online educational game used in the study (c) have no prior 
knowledge of diffusion of innovations theory (the topic taught by the game) and (d) be a native English speaker. 
Students were offered eight dollars per hour for a total of three hours of participation. The majority of the recruited 
students were between 1 8 and 25 years old (93%), were in college (82%), and stated playing digital games from one 
to two hours per week (57%). 

Context of the Study 

This study used an online version of a game called the “Diffusion Simulation Game” (DSG). The DSG 
teaches concepts and strategies related to the diffusion of innovations theory as described by Rogers (2003). Players 
take the role of a change agent working at a junior high school. Their mission is to persuade all school staff 
members to adopt a particular instructional innovation by using the most appropriate activities (e.g., Presentation, 
Demonstration, and Pilot Test) at the right time and with the right staff members. Figure 1 shows the interface of the 
DSG version used in this study, which was based on the original board version developed by Molenda and Rice 
(1979). 
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Figure 1. The Diffusion Simulation Game Interface 

As it can be seen in figure 1, the interface consists of two main sections. The section on the left side 
contains the Information Activities and the Diffusion Activities. Each of these activities has a cost in weeks 
associated with it. After selecting an activity, the weeks are marked off on the calendar, which is included on the top 
of the right section. Players have up to one academic calendar year, which spans from September to June. Below the 
calendar, there is the list of staff members that need to be persuaded to adopt the instructional innovation. 

Once the player gets personal information about each staff member, players can start classifying them by 
their adopter type (which includes Innovators, Early Adopters, Early Majority, Late Majority and Laggards). Each 
staff member needs to get “adoption points” for each of the three stages of adoption (Awareness, Interest, and 
Trial/Appraisal). The small green squares show the “adoption points” that each staff member has earned so far. The 
light green buttons represent the adoption points that were earned in the last turn. The green check mark on the 
“Adopter” column means that the staff member has already become an adopter. 

The version of the DSG used in this study is slightly different to the official online version provided by the 
School of Education at Indiana University. The official online version does not provide multiplayer capabilities and 
for that reason, the first author developed a new version of the DSG that could be played within the three- 
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dimensional virtual world called Second Life. Second Life was selected over other technologies for the following 
reasons (a) Co-browsing capabilities, which allowed all players to see the same screen and share the same progress 
status; (b) Individual control over some browser elements, which allowed players to scroll up and down within the 
game interface and to expand collapsible sections without affecting the interface of the other player; and (c) Voice 
over IP, which allowed players to communicate verbally though their avatars. 

Procedure 

Students’ participation lasted a total of three hours split into two different sessions. Students attended the 
first session individually. In the first session, we (1) measured students’ personality traits, (2) measured students’ 
previous knowledge about the diffusion of innovations theory, (3) collected demographic information about the 
students and game habits, and (4) assigned students to the individual or collaborative setting. Each of these activities 
is explained in the following paragraphs. 

Personality traits were measured using John’s et al. (2008) Big Five Inventory personality test. This 
instmment consists of 44 Likert-style items that measure agreeableness, conscientiousness, extraversion, 
neuroticism, and openness to new experiences. The instrument includes items such as “I am someone who is 
talkative”. The alpha reliability of this instrument is 0.83. 

Student’s previous knowledge about the diffusion of innovation theory was measured by a 15-item pre- 
test, which included conceptual and procedural questions. The validity of this instalment was measured by having 
four graduate students with considerable knowledge about the diffusion of innovation theory taking the test. Fleiss’ 
kappa inter-rater agreement was 0.84. 

A survey was used to collect demographic information about the participants such as gender, age, academic 
level, and game habits. During the first session we also assigned each student to an individual or a dyad setting by 
flipping a coin. We then paired off students assigned to the dyad setting based on a homogeneous level of 
agreeableness. 

The second session took place within a month after the first one. In this session, students (1) played the 
DSG several times for eighty minutes in their assigned setting; (2) took a post-test that measured their gains on 
learning about the diffusion of innovations theory; and (3) submitted an attitudinal survey regarding their reactions 
about having played the game in the setting they were assigned. Students were asked to play an entire game (from 
beginning to end) just the first time. After that, they could restart the game at any time they wanted and play as 
many times within the allotted eighty minutes. 

Students playing in dyads were in different locations, did not previously know each other, and used voice 
over IP to communicate while playing the game online. Based on instructional strategies suggested by the 
cooperative learning literature (Johnson & Johnson, 1991; Kagan, 1991; Slavin, 1995), dyads were provided with a 
script to follow in order to encourage positive interdependence, group monitoring, and individual 
accountability. Moreover, in order to build rapport between the students playing collaboratively, they were given ten 
minutes to play a collaborative game in which each other attempted to identify some of the commonalities between 
them (such as having the same kind of pets, liking the same kind of music, or having watched a specific movie 
recently). The following figure shows the avatars of two participants playing the DSG within Second Life. 
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Figure 2. Participants playing the DSG collaboratively within Second Life 


In order to analyze the interaction between students playing collaboratively and also the interaction of all 
players with the game itself, all game sessions were recorded using screencast software. The software also recorded 
the verbal communication of those playing in pairs. Students in the individual setting were told to think aloud while 
they were playing and their voice was also recorded. The recordings were mainly used to analyze the patterns in 
game play strategies between lower and higher performing players per setting. 

Findings 

In order to address the first research question regarding the relationship between personality traits and 
performance, learning and attitude in students playing individually or collaboratively, we conducted several 
bivariate correlational analyses. 

Personality traits and Game Performance 

Game performance was measured by the average score across all the complete games played by teams and 
individual players. In the individual setting, the only personality trait that was related to game performance was 
conscientiousness, and this relation was marginally significant (r = 0.390, p = 0.073, n = 22). This result was aligned 
with the literature of personality psychology in the sense that people who perceived themselves as highly 
responsible, reliable, and tenacious are associated with improved task perfonnance (Bell, 2007). 

In the collaborative setting we controlled only for the level of agreeableness to pair students. Based in the 
literature of personality psychology, we had hypothesized that teams with high level of agreeableness would 
perform significantly higher. However, contrary to this hypothesis, participants’ levels of agreeableness were 
negatively correlated with their game performance (r = -0.41 \,p = 0.058, n = 22). This indicated that teams in 
which both students perceived themselves as highly polite, tolerant, and trustful had the lowest scores. 

An unexpected finding from the collaborative setting was that the participants’ level of extraversion, which 
refers to the level of sociability, gregariousness, and talkativeness was also negatively correlated to their game 
performance (r = -0.459, p = 0.032, n = 22). This finding was unexpected because the dyads formation did not 
control for this personality trait. This finding indicated that dyads with a higher level of extraversion obtained lower 
game scores than dyads with a lower level of extraversion. 

As we had been predicted it based on the literature of cooperative learning, the average game scores of 
students playing collaboratively was statistically significantly higher than those playing individually, t(31) = 1.71 1, 
p = .0485 {1-tailed). 
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Personality Traits and Gains in Learning 

Gains in learning were measured by the difference between the post-test and the pre-test scores. Both tests 
were taken individually. In the individual setting, the level of extraversion was the only personality trait found to be 
significantly correlated to learning gains (r = -0.478, p = .025). This correlation was negative, which indicated that 
students who perceived themselves as more introverted tended to gain significantly more knowledge than those who 
were more extraverted. 

For participants in the collaborative setting, there were two personality traits that were marginally 
significantly correlated to their learning gains: conscientiousness (r = 0.37,/? = 0.09, n = 22), and openness to 
experience (r = 039, p = 0.073, n = 22). This result indicated that participants who perceived themselves as more 
responsible, reliable, and creative obtained higher gains in learning 

Personality Traits and Students’ Attitudes 

Participants’ attitudes towards their learning experience were measured by a reactionnaire that included 
several Likert scale items and an open-ended question. The Likert items were divided into two sets. The first set 
consisted of six items measuring the participants’ level of enjoyment playing the DSG. It included items such as “I 
enjoyed Playing the Game” and “If given the chance, I’d like to play the game again”. The Cronbach’s alpha 
reliability of the scale measuring the level of enjoyment was 0.805. 

The second set of items was used to measure the extent participants liked playing in the setting they were 
assigned, either individual or collaborative. These questions varied slightly per setting. For instance, for participants 
playing collaboratively, one of the items read: “I felt more motivated playing the DSG as part of a team than if I had 
played alone”, whereas for participants playing individually, the corresponding item read: “I feel that I would have 
been more motivated playing this game with somebody else instead of playing it alone.” Some of these items were 
adapted from an attitudinal survey used by Brewer and Klein (2006). 

On average, regardless of the setting they were assigned, most participants agreed having enjoyed playing 
the game, wanting to play more, and considered that playing the game was an effective way to learn. In the 
individual setting, the only personality trait that was statistically correlated to the participants’ attitudes toward 
having played the game individually was their level of agreeableness (r = - 0.421,/? = 0.051, n = 22). This result 
indicated that students who did not consider themselves very cooperative and supportive to others, preferred playing 
the game alone. Students playing in teams exhibited a significant correlation in their level of extraversion ( r = 0.567, 
p = 0.006) and neuroticism (r = 0.466 ,p = 0.029). This finding indicated that students who perceived themselves as 
more sociable, talkative, and emotionally stable tended to prefer playing the game in a team. 

Common Patterns in Game Play Strategies per Setting 

The second research question involved investigating common patters in the game play strategies used by 
students with the highest and lowest scores per setting (individual and collaborative). The recordings of all the 
games played by the students with the top two and the lowest two average scores were transcribed. The game scores 
were calculated based upon the average number of “adoption points” obtained across all complete games played. 
Table 1 shows the total number of games played by these students and their average game score. 

Table 1. 

Average Scores for Top an d Lowest Two Teams and Individual Players 



Teams 

Total Games 
Played 

Complete 
Games Played 

Average 

Score 

Top 2 
Teams 

Team 8 

4 

4 

9.15 

Team 4 

9 

3 

8.91 

Bottom 2 
Teams 

Team 10 

6 

3 

7 

Team 11 

6 

3 

6.93 


Top 2 
Individuals 

Player 16 

4 

3 

9.24 

Player 11 

9 

3 

9.23 

Bottom 2 
Individuals 

Player 13 

5 

3 

5.01 

Player 22 

6 

3 

4.31 


Analysis of Patterns in Time (APT), which is a method of recording and quantifying temporal relations 
about observable phenomena (Frick, 1990), was used to identify patterns between players with the highest and 
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lowest scores in both settings. In APT, patterns are identified by counting the occurrences of multiple states that 
characterized the phenomena or event observed (Frick, 1983). A set of states that are mutually exclusive and 
exhaustive comprises what Frick (1983) defines as a “classification” in APT. 

Some of the APT classifications used for the analysis of patterns in the collaborative setting were adapted 
from the literature in interaction and discourse analysis (Gunawardena, Lowe, and Anderson, 1997; Tiara, Bonk, & 
Angeli, 2000; Flenry, 1992; Herring, Kutz, Paolillo, & Zelenkauskaite, 2009), and some others were observed when 
analyzing the game plays. Table 2 condenses these APT classifications. 

Table 2 


List of classifications used for APT analysis in the collaborative setting 


Classification 

Description 

Collaboration level 

Extent to which team mates collaborated in deciding the activity to 
conduct in a turn. 

Decider 

Player(s) who made the decision to conduct the activity in the turn 

Suggestions 

Player(s) who made suggestions during the turn 

Turn Reason 

Reason why dyads selected the activity conducted in each turn. 

Diffusion Strategy Selection 

Whether the activity conducted was appropriate according to diffusion of 
innovations theory (i.e., in order to succeed in the game). 


Each of the classifications shown in Table 2 had a set of categories associated with them. For instance, the 
categories associated with Collaboration Level are included in Table 3. 

Table 3 


Categories in the Collaboration Level classification 


Category 

Description 

Negotiation (N) 

The activity is conducted as a result of dialog or discussion between players 
that includes sharing information, elaborating a suggestion, expressing 
different opinions, participating in the decision process, and identifying 
inconsistencies. 

Passive Negotiation (PN) 

A player suggests conducting an activity and the other player consents without 
any dialog. The suggestion is usually presented as a question. 

Unilateral Decision (U) 

In his/her turn, a player conducts an activity (which was not previously 
discussed) without the other player’s opinion. 

Command (C) 

In the other player’s turn, a player commands him/her to conduct an activity. 


Each game turn of all complete games played by the top two and the lowest two teams and individual 
players were coded using the categories for each of the identified APT classifications. A second rater analyzed and 
coded ten percent of random selected turns independently in order to calculate inter-rater reliability. On average, the 
Cohen’s Kappa coefficient across all APT classifications was 0.73. 

Based on the APT analysis, it was possible to observe that in both settings, the main differences between 
the higher and lower performing dyads regarding the reasons for conducting an activity were that the higher 
performing dyads conducted more activities as a result of an externalized cognitive process and of applying what 
they have learned from previous turns. For example, on average, higher performing dyads conducted 1.85 times 
more activities that were result of a cognitive process and 1.76 times more activities that had been previously 
identified as being effective, compared with low performers. Moreover, in terms of the reasons for conducting 
ineffective strategies, the lowest performing dyads conducted 2.5 times more unsuccessful activities for failing to 
observe information or hints on the feedback provided. 

By counting the number of occurrences in which game turns had been coded as “Negotiation”, it was 
observed that, on average, the higher performing teams had conducted almost twice as many negotiated turns than 
the lower performing teams. On the other hand, lower performing teams had 1 .42 more turns using passive 
negotiation and had more than twice as many turns that used unilateral decisions. 

Regarding the number of suggestions made by players, through APT it was observed that on average, 
compared to the lower performing teams, higher performing teams conducted 1.84 more turns in which both players 


101 




made suggestions. In contrast, lower performing teams conducted 1 .43 more turns in which neither player made any 
suggestion. 

In the individual setting, participants with the highest average score across all their games conducted 
effective turns that were result of an externalized cognitive process 2.6 times more often than participants with the 
lowest average score. Moreover, higher performing individual players conducted twice as many activities that had 
been previously identified as being effective. Lower performing individual players were over three times more likely 
to repeat non-effective activities due to having forgotten information or feedback about the previous activities 
conducted. 

Conclusions 

In this study we explored (1) how personality traits correspond with performance, learning, and attitude in 
students playing the DSG individually and collaboratively and (2) pattern differences between higher and lower 
performing players. 

In partial agreement with personality psychology literature, conscientiousness was found to be marginally 
significant to game performance in the individual setting (r =.390. p = .073, n = 22). However, contrary to 
expectations, in the collaborative setting the levels of extraversion and agreeableness were negatively correlated to 
their game performance (r = -0.459,/? = 0.032, n = 22 and r = -0.411,/? = 0.058, n = 22 respectively). 

When playing the DSG, dyads with a lower level of agreeableness exhibited more negotiation in each game 
turn whereas those with a higher level accepted passively more suggestions put forward by their teammates. In 
general, dyads with a lower level of agreeableness had more cognitive disagreement, corrected each other’s mistakes 
more often, and offered more suggestions. 

Regarding the relationship between personality traits and gains in learning, extraversion was found to be 
negatively correlated in those playing individually (r = -0.478,/? = 0.025, n = 22). This finding indicated that, 
regardless of their game performance, individual participants who perceived themselves as more introverted and 
reserved obtained higher gains in learning. An analogous correlation was not found in the collaborative setting. In 
other words, extraverts playing in dyads did not exhibit greater gains in learning. 

Pattern differences were analyzed using APT. In both settings, a relevant difference between the higher and 
the lower performing participants was the number of turns that were conducted as a result of a cognitive process 
externalized by either thinking aloud, in the case of the individual setting, or by talking to a teammate, in the case of 
the collaborative setting. This difference was more pronounced in the individual setting, in which higher performing 
participants conducted 2.6 times more cognitive -related turns than lower performing participants. 

In addition, in both settings, lower performing participants conducted ineffective turns more often than 
higher performing participants. On average, they conducted 2.5 times more turns that were ineffective for failing to 
notice information or hints on the feedback provided in the game. 

In the present study, it was found that participants who were more actively involved in the decision-making 
process and who had a higher frequency of communication, exhibited greater gains in learning. Tsay & Brady 
(2010) reported a similar finding after observing that students participating more actively in a team-based learning 
course had also obtained higher scores. Ensuring a more active participation by all team members could thus be 
desirable in synchronous collaborative educational games. 

Limitations 

A main limitation of this study is not generalizable to any other contexts. The study used a specific 
problem-solving activity that consisted of using the Diffusion Simulation Game as the educational game instance. 
Each educational game has a different level of complexity, underlying instructional information, rules and 
mechanics, and expected outcomes. 

A second limitation is its relatively small number of participants. The study compared two groups that 
included only 22 participants each. Increasing the number of participants per group could have also increased the 
statistical power of the study. 

An additional limitation is that the study did not measure long-term learning. The participants took the 
post-test immediately after playing the DSG during an 80-minute period. They did not take another follow up test 
weeks later. Such a test would have allowed comparing long-term retention based on the setting assigned. 
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Competitive higher education institutions strive to attract, enroll, and retain learners with online courses. Studies 
have revealed that online learners may select courses for access and convenience although, once enrolled, they value 
specific course features and conditions (Dennen, Darabi, & Smith, 2007). Online learners do not value all 
instructional features and conditions equally nor do all instructional features and conditions help them leam equally 
well. This paper suggests learner-valued online course features and conditions synthesized from studies in social 
presence (Wei, Chen, & Kinshuk, 2012), feedback (Espasa, & Meneses, 2010), and learner self-regulation (Ley & 
Young, 2001). A seminal study that identified learner-valued interactions (Dennen, Darabi, & Smith, 2007) when 
compared with findings from studies in social presence, performance feedback, and learner self-regulation frames 
online course design that transforms learner-valued features into effective instructional strategies. 

College and university online course design has to address effective learning strategies but also must address 
learner-valued course features given an increasingly competitive higher education market place. Higher education is 
a market place in which institutions compete for students, and academics study the phenomenon as evidenced by the 
Journal of Marketing in Higher Education. Marketing research findings reflect what most administrators 
acknowledge: Students are attracted to online courses for a number of reasons including convenience and course 
availability (Allen, & Seaman, 2010). Most educational organizations now offer online courses to increase 
enrollments, but must consider political, fiscal, or social forces in order to do so. In the competition for students, 
most institutions of higher education are offering more online courses and are converting many on-ground courses to 
online. Many on-ground courses can be adapted to online but it is only with care and attention to learner 
motivational and instructional requirements that an online course provides an appealing and effective educational 
experience. Effective online courses especially in highly competitive online programs encourage online course 
completion and retention. Therefore educators seeking to sustain or increase course and program enrollments are 
rightfully concerned with the online learner’s experience. Instructional effectiveness drives online course design but 
instructional appeal encourages course participation and successful completion. Online courses that engage students, 
encourage them to complete a course, and enable them to earn a passing grade, foster online course and program 
success. 

Instructional designers and faculty seeking to translate theory and research into practice have an overwhelming 
plethora of rich and varied of studies investigating a multitude of online course features and conditions. This 
extensive body of online studies and the theories begs the question, how to design online courses that encourage 
students to engage in effective learning activities? One answer considers what learners value and then compares 
these valued features and conditions strongly supported by the findings of online learning research. What online 
learners value drives the process but instructional effectiveness determines how to design and deliver effective 
learning support with learner-valued interactions. 

A seminal study describing what online learners value, surveyed one hundred and seventy online students from 
two universities and asked them to rank how much they valued 16 instructor activities (Dennen, Darabi, & Smith, 
2007). The top five most-valued instructor interactions were 1) check email to assess learner needs, 2) posted 
discussion board, 3) provide examples, 4) provide timely feedback, and 5) respond to student inquiries. In sum, 
students valued timely, responsive, instructor interactions. Learner-valued instructor interactions signal to online 
students that an instructor is present, available, and helpful. Four of the five valued instructor interactions implicitly 
involve instructors communicating with students; the fifth, provide examples, could be presented through instructor 
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communications or in instructional materials. Three of the valued instructor activities involve individual learner- 
instructor interactions: check email to assess learner needs, provide timely feedback, and respond to student 
inquiries. Learners value instructor messages on a class discussion board whether responding to a specific learner or 
initiating the communication. 

Data supports the conventional wisdom that learners value instructor interaction (Boston, Ice, & Gibson, 2011, 
Reio & Crim, 2013). Research has indicated that social engagement, establishing communities of learning, a 
supportive environment that embraces diverse cultures and demonstrates respect for knowledge (whether online or 
on-ground) are keys to students’ successful attainment of their learning goals (Bonk, 2006; Darling-Hammond, 

2006; Jaschik, 2010; Liu, Magjuka, & Lee, 2008; Patterson, & McFadden, 2009; U.S. Department of Education, 
2009). An instructor’s social presence can support the keys to a positive learning environment. Social presence may 
be defined as “an individual’s ability to demonstrate his/her state of being in a virtual environment and so signal 
his/her availability for interpersonal transactions" (Kerhwald, 2010, p. 94). Instructor social presence, as measured 
by student perceptions of his/her interactions with the instructor, has contributed to learner satisfaction (Boston, Ice, 
& Gibson, 2011; Cui, Lockee, & Meng, 2013). In fact, social presence was one of four extrinsic factors affecting 
online learner satisfaction in a data-based model of online learner satisfaction (Lin, Lin, & Laffey 2008). In the 
model, social presence was the extrinsic factor over which instructors had the most control. 

A structural equation model (Wei et ah, 2012) based upon responses from over 500 high school and 
undergraduate participants responding to a self-report measure indicated that social presence positively influenced 
learning interaction that in turn influenced learning performance. A case study comparison between the highest-rated 
instructor and the lowest-rated instructor revealed that instructor interaction frequency between the two courses 
differed dramatically (Gorsky & Blau, 2009). While frequency of interaction was important, it was not the only 
determinant of learner-valued interactions. The content of the instructor-student interactions was also important; 
students wanted more than “I agree”, “good response”, or “nice job.” Learners wanted more insightful instructor 
feedback. Furthermore instructor interaction frequency appears to have diminishing returns (Dennen, Darabi, & 
Smith, 2007). 

Feedback, a fundamental and essential learning component, enables the learner to monitor and evaluate her own 
learning processes, two of four components in an instructional design model for embedding self-regulation (Ley & 
Young, 2001). Self-regulation may be defined as one's ability to plan, organize, monitor and evaluate his or her own 
learning (Ley & Young). The self-regulated learner monitors his or her learning processes and is able to evaluate 
how effective those learning processes are. Instructor interactions may be most important with poorly self-regulating 
learners because they are less likely to benefit from online student-content interactions (Bol & Garner, 2011). On the 
other hand, online courses must be motivating even for the self-regulated students (Howland & Moore, 2002). 
Instructors who attempt to provide learners with valued instructor interactions inherently match learner’s motives, an 
instructional strategy that also is an instructional motivational strategy (Keller, 2010). Feedback has been 
significantly related to online learners’ affective and cognitive learning outcomes as measured by students’ 
satisfaction and grades (Espasa, & Meneses, 2010). For some learners more feedback may be required than for 
others (Howland & Moore). Learners who watched instructor feedback podcasts commented they felt connected to 
the instructor and some preferred the podcast feedback to print (Bolliger, Supanakom, & Boggs, 2010). 

Online instructor communications can simultaneously address learner -valued interactions and self-regulation, 
feedback, and social presence. A comparison of valued interactions with each of these constructs indicates which 
learner-valued interactions have potential to promote self-regulation, provide feedback and establish instructor social 
presence (Table 1). Each of the five most valued instructor interactions should also be vehicles for one or more of 
proven instructional methods associated with these constructs: reinforcing self-regulation, providing learner 
feedback, and establishing instructor social presence. The prudent online instructor can leverage learner-valued 
interactions into useful feedback and self-regulation support that also reinforces the instructor’s social presence. 

Each of the five top learner-valued interactions can be a vehicle for delivering one or more of these effective online 
instructional methods. A description integrating each interaction with the three instructional methods illustrates 
possible instructor approaches. 


105 



Table 1. Learner-Valued Instructor Interactions Support Effective Learning 



Social 

Presence 

Self- 

Regulation 

Feedback 

Check email to assess learner needs 

x 

x 

x 

Posted to discussion board 

x 

x 

x 

Provide examples 

x 


x 

Provide timely feedback 

x 

x 

x 

Respond student inquiries 

x 

x 

x 


Check Email to Assess Learner Needs 

Checking learner emails is an excellent way to build social presence, provide feedback, support learner self- 
regulation, and ultimately motivate learners. Instructor responses to learner emails reinforce social presence and can 
also encourage learner self-regulation if the instructor’s communication is timely and includes feedback about the 
learner’s academic performance. When an instructor responds may be as important as the response itself. A one to 
two day time frame is usually considered sufficiently prompt to be perceived as timely by the learner but may be 
difficult unless coupled with syllabus policies and course features that support response timeliness. Email logistics 
can be overwhelming when an instructor has large online course enrollments unless an instructor has syllabus 
policies which mediate and moderate email communications (Ley, in press). 

Where and when an instructor responds to students affects how well the instructor’s response reinforces his 
positive social presence. One discussion board message viewed by the class promotes social presence among the 
entire class; one email response to an individual learner is a uniquely personal interaction. To convey a positive 
instructor social presence in an individual email requires additional semiotic considerations besides timeliness. It 
requires the rhetoric of positive social presence, such as, thanking the student for initiating contact, beginning with a 
personal greeting, and expressing concern for the student’s learning. Email communications that incorporate 
personal messages reinforce positive social presence. 

An instructor may find it far easier to respond in a timely fashion if fewer emails require attention; replacing email 
with class communications is an option with other benefits as described in the next section. 

Post To Discussion Board 

Where and when an instructor responds to a learner's questions are no less important than what the instructor 
communicates. To convey a positive instructor social presence requires additional semiotic considerations besides 
timeliness. It requires the rhetoric of positive social presence; thanking the student for initiating contact, beginning 
with a personal greeting, and expressing concern for the student’s learning reinforce an instructor’s positive social 
presence. Whenever appropriate an instructor should attribute a learner’s performance to effort since effort 
attributions correlate positively with academic performance (Cleary & Zimmerman, 2004). 

Encouraging discussion board or other class communication venues has advantages over email for online 
instructors and students. The very act of posting a message is a learner-valued instructor interaction and promotes 
social presence. At the beginning of a semester an instructor can post an introductory message on the discussion 
board. Evidence suggests that audio is especially preferred for online course communications (Bolliger, Supanakorn, 
& Boggs, 2010). An instructor who answers questions about future assignments and who posts class feedback about 
past assignments builds social presence. An instructor can post messages reminding students of future assignment 
due dates or suggest milestones toward completing assignments. Therefore an instructor’s class discussion board 
messages are also an opportunity for instructors to promote their social presence, provide feedback, and promote 
learner self-regulation. 

Provide Examples 

Providing examples is a powerful learner-valued interaction because it is also an effective instructional strategy 
(Huh, 2013). Learners value examples because it can be very difficult to describe in a few words what the desired 
outcome should be. Frequently it is much more powerful, effective, and motivating to display an example. Examples 
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represent the required performance. Examples symbolize what the instructor is expecting in an assignment. 

Examples before an assignment illustrate the instructor’s expectations. Examples after an assignment provide 
feedback and may be more appropriate where an example prior to an assignment discourages original critical 
thinking. 

Provide Timely Feedback 

Timely feedback is as essential to learning as it is to providing a learner-valued interactions. Effective feedback 
does more than just provide knowledge of results. Knowing whether one got the right answer or not is often not 
sufficient to help the learner correct misconceptions or improve work. Feedback can tell the student where and how 
the assignment meets or does not meet the assignment criteria or is inconsistent with the instructions. The most 
detailed level of feedback describes how to correct the work or performance. One of the best ways to provide 
feedback may also be one of the most efficient. Most instructors soon discover that audio feedback can be produced 
with far greater detail and faster than with print feedback. An instructor can embed audio comments into written 
assignments, record audio or video feedback for each learner, or embed print comments in a document. 

Rhetoric for feedback is also a consideration. Referring consistently to an assignment rather than the student 
sends a message that it is the assignment and not the student that may need improvement. This rhetoric enables the 
instructor to help the student instead of implicitly disapproving of the student. Whenever prudent, an instructor 
should attribute learner’s performance to effort since effort attributions correlate positively with academic 
performance (Cleary & Zimmerman, 2004). Well-done work or performances attributed to effort signals the 
instructor’s confidence in the learner. On the other hand learners who profess to have worked hard should receive 
feedback about how to improve work since they may be working hard but on the wrong learning activities or study 
strategies. Feedback on learning processes addresses learner self-regulation by helping learners who may not be able 
to evaluate their learning processes or identify effective alternatives. Timely feedback can support self-regulation 
and promote an instructor’s social presence. 

Respond To Student Inquiries 

The fifth most valued instructor interaction was responding to student inquiries. This instructor interaction 
specifically refers to a learner receiving a response to questions the learner has posed to the instructor. Responding 
to inquiries could occur in email, on the discussion board, or even by telephone. If on the discussion board or in 
email the message inherently promotes the instructor's online social presence. The nature of the inquiry determines 
which of the two other functions the response could serve. A learner’s question may present an instructor with the 
opportunity to offer a learner self-regulation support that suggests to the learner how to plan, organize, monitor, and 
evaluate his own learning. If the student’s inquiry is about a test answer or the instructor’s evaluation of the students 
work, the instructor has the opportunity to provide feedback such as is described in the timely feedback section. 

Conclusion 

Online instructors have an opportunity to leverage learner-valued interactions into effective learning support. 
Research results often report the relationship between a single construct and perhaps student perceptions or, in some 
cases, learning outcomes. By considering how to integrate important learning support with interactions that online 
learners most value the online instructor makes each message a valued learner interaction that can also build the 
instructor’s positive social presence, provide useful feedback to learners and promote their self-regulation. 
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Abstract 

The flipped classroom, often referenced as “flipping the classroom”, is a teaching and learning model in which the 
students spend a short period of time learning the basic concepts outside the classroom prior to the introduction of 
the concept topic in the classroom setting, and then use class time for authentic experiences such as problem solving. 
This action research study described the implementation of a collaboratively-designed, flipped classroom 
instructional model in a large-enrollment introductory Genetics class for undergraduates. The aim of implementing 
the flipped classroom model in this class was to improve students’ learning motivation, enhance the students’ ability 
to apply knowledge, and improve the teaching and learning efficiency by transforming the traditional lecture into a 
more interactive instruction. This paper provided information for instructional designers and higher education 
instructors about the general framework of introducing the flipped classroom strategies in a large introductory 
biology class. The purpose of the paper is also to share the instructor’s and students’ perceptions towards the new 
learning approach, their performance in teaching and learning, and lessons learned in the role of instructional 
designer and teaching assistant. 


Introduction 

The term “the flipped classroom” was first put forward by Jonathan Bergmann and Aaron Sam in 2006 
(Makice, 2012). Bergmann and Sam are chemistry teachers in Colorado who pioneered using screen casting and 
video podcasting to deliver the learning content for their science courses in high school. The flipped classroom has 
since become a teaching and learning model in which the lecture content or background informational material is not 
presented in the classroom but is experienced by students through various and alternative forms of media prior to the 
classroom instructional session in which the material is addressed. This frees the valuable class time is allowed for 
active learning, and the content that was removed is delivered to students via online video or other formats (Baker, 
2000; Zappe, Messner, Litzinger, &. Lee, 2009). 

Different from traditional lectured-based instructional approaches, the two main phases of instruction are 
reversed, or “flipped.” In the traditional lecture -based instructional approach, the first phase is the classroom 
presentation of the learning content. This is typically followed by a second phase, which is practice, memorization, 
or application of the knowledge to new settings. In the flipped classroom approach, the presentation of learning 
content is done before the class meeting through various media. Then the classroom part becomes a setting for 
various types of active learning, such as problem solving, laboratory experiments, collaborative designing and 
creating projects, happens during the class time (Gerstein, 2011). The class meeting is not for the introduction of 
new learning content but rather a where the introduction is elaborated in some meaningful way. 

The fundamental idea behind flipping the classroom is that more classroom time should be devoted to 
active learning and that the teacher can provide immediate feedback and support for learning during the classroom 
session (Warter-Perez & Dong, 2012). The flipped classroom approach is a flexible set of teaching and learning 
strategies for organizing student learning experiences in a manner that requires active student engagement 
throughout the learning process, rather than a specific educational approach (Makice, 2012). In recent years, videos 
and podcasts have been used to stimulate the students’ interest in pre-class basic concept learning (Demetry, 2010; 
Gerstein, 2011). Various ways can be used in the two phases of the flipped classroom model, the pre-class phase and 
the in-class phase, to start with student effort before a class meeting and engage students in active learning activities 
in class. 

The flipped classroom model can be adapted to many different classroom environments (Warter-Perez & 
Dong, 2012). Creating a learner-centered active learning environment in a large introductory class by replacing the 
classroom focus on lectures or information presentation to students with active learning activities can increase 
students’ engagement, critical thinking skills, and interests (Moravec, Williams, Aguilar-Roca, & O'Dowd, 2010). 
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Nevertheless, the need for immediate teacher feedback in the classroom part of learning provides challenges to the 
instructor to introduce the flipped classroom and other student-centered active learning models in large classes 
(Pastirik, 2006), and the teacher would have less time to provide individual support (Arias & Walker, 2004). 
Moreover, in large classes, not all the students like active learning, and it is difficult for the instructor to pay close 
attention to each student’s performance and engage all the students in learning (Bates, 2005). Instructors can have 
various interventions to promote students’ engagement in active learning. These interventions include peer 
instruction (Lage, Platt, & Treglia, 2000), “re-shaping” the large class into small groups, and introducing hands-on 
activities and problem solving in class (Tolley, Johnson, & Koszalka, 2012), etc. 

This study examined the process of introducing a flipped classroom model in a large-enrollment 
undergraduate introductory biology class. This study had several purposes: (a) presenting the general framework of 
the flipped classroom in a large undergraduate biology class to produce a student-centered active learning 
environment; (b) examining the students’ and teacher’s performances during the whole class; (c) determining the 
students’ and teacher’s perceptions towards the flipped classroom model. 

Background 

The course 

The course selected for this study addressed the topic of “General Genetics.” It was an introductory level 
course for the undergraduate students in biology and biology related majors such as microbiology, biomedical 
engineering, and plant science. The course objectives focus on the principles of heredity, meiosis and transmission 
genetics, DNA/RNA structure, gene expression and regulation, mutation and genome structure, and population and 
evolutionary genetics. This course is also a required course for the students who wish to apply for medical school. 
The course was delivered in face-to-face format without significant use of outside-of-classroom electronic 
communications media for instructional interactions among students, teachers and teaching assistants. There were 
three classes per week, and each class section lasted for 50 minutes. The classroom was a traditional lecture hall 
with the capacity of 150 students. 

Participants 

A total of 120 students who registered for the course participated in this study. All were from biology 
related majors. Five of the students planned to apply to medical school. Most students were sophomores and juniors. 

The instructor was of Chinese ethnicity and worked as an associate professor in biology in a large research 
university in the Southeastern US. He had been teaching this course for more than ten years. Prior to this study, he 
expressed the desire to introduce something new in his course to motivate the students’ learning, but still adhered to 
the traditional lecture teaching method and the same teaching materials for many years because he had “no time” 
and “no idea” for change his teaching format. The pressure of research made him have little time and energy to 
devote into undergraduate instruction, and the lack of educational knowledge made him feel more confused and 
challenged on improving his instruction. This study was proposed as a way to implement a new teaching strategy to 
help him attain his goals, and would provide data to help him determine the efficacy of the flipped classroom 
instructional approach. 

The researcher was a Chinese doctoral student in instructional technology in the same university with the 
instructor. They collaborated to design and implement the flipped classroom approach described in this paper. The 
researcher played two roles in this study, that of instructional designer and teaching assistant in this course. 

However, the researcher had no biology background, so she could only assist the instructor on the issues related to 
instructional design and answering the students’ questions about the learning activities. It was the first time the 
researcher had designed and implemented active learning activities independently, and also the first time she had 
communicated with so many American undergraduate students in the role of teaching assistant. 

The problem 

The instructor was concerned with two problems. First, in his experience of teaching this course for more 
than ten years, he found it was a common problem that the students never read the textbook, and did not know how 
to read and teach themselves from scientific writing. The textbook, “Introduction to Genetics Analysis” (Griffiths, 
Miller, Suzuki, Lewontin, & Gelbart, 2010) had been updated for 10 editions, but most knowledge the course 
covered, such as chromosome structure, and DNA/RNA synthesis, remained the same. In the first meeting with the 
instructor, he told the researcher “it is impossible to absorb all the information in the textbook, but the students don’t 
know which they should focus on when they read.” Without a basic knowledge of genetics, the students faced 
enormous difficulties in understanding the class lectures, and the instructor had to spend large amount of class time 
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illustrating the basic concepts. As a result, the instructor felt that the students’ learning was not optimal. He was 
tired of lecturing to the students and sensing that they were not actively involved in learning. 

The second problem that concerned the instructor was the time issue. He had heard of more student- 
centered learning models implemented by some other professors in the College of Engineering, but he worried it 
would take too much time to learn the educational knowledge, and design totally new course materials, in order to 
introduce new teaching and learning approaches into his course. As a result, he concluded that he would not be able 
to implement a new instructional model in his course. 

New practice 

The researcher and the instructor worked collaboratively to develop and implement the flipped classroom 
model. This model required three major changes to the pre-existing course. 

The first change was adding a “reflective journal” learning activity in the course, in order to help the 
students focus on pre-class reading and be better prepared for the in-class activities, and also to motivate their 
higher-order thinking. 

At the beginning of learning each new chapter, the researcher made an orientation to the students about the 
requirement of the reflective journal activity. The students were required to submit a pre-class reflective journal 
about what they had learned in reading the chapter topics, such as key points outline, summary, understanding, 
discovery, and confusion, etc. They were encouraged to write in their own words, share their own understanding, 
and relate what they gained from reading with real life experiences. After the class activities of each topic, the 
students were required to revise the pre-class reflective journal in another color font and submit it for credit as the 
post-class reflective journal. They were encouraged to correct the misunderstandings in the pre-class journal, write 
down the answers to the difficult questions put forward in their pre-class reflective journals, and add something 
more on understanding about the learning content. During the course, the students had to finish 12 reflective 
journals, but only 5 of them were selected for grading. The students were only told that 5 of the journals would be 
randomly selected to grade, and they should do their best in working on each journal because it may be calculated 
into their final grades. 

The second change was that the amount of in-class lecture was reduced. More class time was spent for 
student-generated Question and Answer and group discussions about genetics problems. The instructor had more 
time to work on Genetics problems with the students. 

The third change was that collaborative projects were added in the course learning. The students were 
divided into groups of 10 students. Each group was assigned a Genetics topic related to real life, for example, 
genetically modified food, blood type, and Thalassemia. The students were required to collaborate on the assigned 
topic, and give a presentation to share their findings with the classmates. 

In order to motivate the students to collaborate, the students were informed that all the students in the same 
group would share the same grade, so they should try their best to improve the effectiveness of their teamwork. 
However, in order to avoid the phenomenon where only one or two students in a group did most of the work but all 
others “shared” the grade without making contributions, the researcher designed a peer assessment for the students 
to monitor each other’s contributions to the group. All the students were also notified that the peer assessment was 
not to make them compete with each other, but to provide a mechanism for them to report on the collaborative effort 
of other group members. 


Method 

Data collection 

To determine the effectiveness of the flipped classroom model used in this class, the researcher chose 
action research as the methodology of this study. Action research would enable the researcher to plan the learning 
activities and make modifications as the model was implemented (Koshy, 2010). Moreover, reflection is a key 
component of the action research process (Mertler, 2009). In this study, the researcher’s reflection played an 
important role in designing and modifying the learning activities, communicating with the instructor, students, and 
other people involved in this course. 

Students’ free responses in their pre-class and post-class reflective journals were selected as one of the 
sources of data for analysis. According to the requirements for the reflective journal, in their pre-class reflective 
journals, the students should write their outline of key points in each chapter, along with the summary of what 
they’d read, their personal understandings, discoveries, and misunderstandings. When they worked on revising their 
pre-class reflective journals after class, they were required to expand their understandings and reflections about the 
chapter, and the revision of their former understandings or misunderstandings. They students were also required to 
simply state what they liked and disliked about the reflective journal learning activity. 
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In the first reflective journal, lots of students misunderstood the requirement. Some even submitted several 
sentences such as “I understand this chapter, it’s easy”, “I’ve learned it in high school”. The researcher spent lots of 
time grading the students’ first reflective journal and gave them individual feedback. After the grade was released, 
the researcher received lots of questions about grading. What concerned the students most, especially those planning 
to apply for medical school, was how to get a higher grade. GPA was very important to them, and they worked very 
hard to complete the assignments. 

Considering the large number of misunderstandings and general confusion about the first reflective journal, 
the instructor and the researcher decided that the grade on the first reflective journal would not be calculated in the 
final grade. Moreover, the researcher made the reflective journal guideline more detailed. The researcher required 
the students to write their pre-class reflective journals according to the structure of “outline”, “summary”, 
“understanding”, and “discoveries”. The researcher also informed the students that if they had nothing to say in 
some of the four parts, just keep them blank. In order to get students’ feedback and improve the instructional design, 
the researcher suggested that the students provide feedback at the end of their journals if they had suggestions about 
instruction and the learning activity. 

The instructor and the researcher arrived at consensus on selecting five reflective journal assignments for a 
better and broad ranging set of responses across the whole semester. The students’ responses on sharing 
understandings and discoveries were selected because their independent thinking and higher-order thinking skills 
could be seen through sharing their personal understandings and discoveries in their reading journals. 

The researcher acquired data about the students’ participation and contribution to the collaborative projects 
from the peer assessment. Additionally, following the collaborative projects, an online anonymous survey was 
administered to the students. The survey contained rating scale items and open-ended questions designed to discover 
how the students searched the material, how they explored, how they collaborated with other group members, and 
how they liked the collaborative project learning activity. 

The researcher’s journal was also an important source of data collected in this study. It was the first time 
for the researcher to design and facilitate the activities in a science course, and it was the first time for the researcher 
to work with a professor to transform the traditional lecture teaching model into student-centered learning model. In 
the journal, the researcher recorded all the instructional design plans, and modifications, together with all the 
communications with the instructor, the students, and other people who provided support. The researcher’s journal is 
also the record of the researcher’s reflection, which is important in action research (Mertler, 2009). 

Data analysis 

Students’ reflective journals were analyzed to gain the students’ understandings on the genetics topics. Data 
analyses were conducted on three variables related to the pre-class reflective journal responses. The first variable, 
“submission rate”, was defined as the percentage of the students completing the pre-class reflective journals, i.e., did 
the student complete the pre-class reflective journal ahead of the classroom activity as scheduled. The second 
variable, “share understanding”, was defined as the percentage of students who interpreted their personal 
understanding in their own words in their pre-class reflective journals. The third variable, “share discovery”, was 
defined to be the percentage of the students whose per-class reflective writing demonstrated making connections to 
real world problems, hot spots in biology, and laboratory experiences in other courses. 

For the closed-ended questions in the survey, descriptive analyze determined students’ perceptions and 
values on the collaborative project learning activities. For the open-ended questions in the survey, document analysis 
was used to determine the students’ key ideas about collaboration and suggestions to improve learning efficiency. 
The researcher’s journal recorded the information from the instructor and two biology graduate students about their 
efforts and attitudes related to the implementation of the flipped classroom model in the genetics class. 

Results 

The sources of data described above were analyzed to determine the impact of the flipped classroom 
instructional approach on student’s attitudes, behaviors, and understandings of genetics ideas. The following are the 
interpreted results. 

Findings from reflective journals 

The students were required to submit 12 reflective journals during the semester. The five reflective journals 
selected for data analysis were the 3rd, 4th, 7th, 9th, and 11th reflective assignment the students submitted. In the 
3th reading journal, 100 of the 120 students submitted the pre-class reflective journal on time. However, the students 
did not perform very well on deep thinking and presenting their own thoughts as demonstrated by the lower number 
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of “share understanding’’ and “share discovery” students included in their writing. Among the 100 students who 
submitted their pre-class reflective journals on time, only 54 (54%) “share understandings”. The other students’ pre- 
class reflective journals resembled an outline of key points in the textbook (Figure 1). Only 17 (17%) students 
among the 100 students included “share discovery” in their writing (Figure 2). The others constrained their writing 
specifically addressed in the chapter of the course textbook. 

Figure 1. Example of the pre-class reflective journal in the outline style 

Reading Journal: Chapter 3 

1. Mendel’s law of independent assortment 

a. Dihybrid crosses 

1) A/a B/b x A/a B/b 

2) 9:3:3: 1 ration in F2 generation 

b. Mendel’s second law 


Figure 2. Example of “share discovery” in pre-class reflective journal 

Reading Journal: Chapter 3 

Outline 


Summary 


Understanding 


Discovery 

It was very interesting to be able to apply Mendel’s second law to real-life scenarios. An 

example is that hybrid crops tend to be better quality than the homozygote dominant or recessive 

It is also due to Mendel’s Law of Independent Assortment that children have different genotypes 

than their parents 


Note: The emphasis is on the “discovery” section, but the student also contributed in the other three sections. 

Students’ submission rate, “share understanding”, and “share discoveries”, is shown in Table 1. Although 
there were some inconsistencies overtime, students’ writing showed increases across the three variables over the 
course (Table 1). 

Table 1 


Students ’ responses in the pre-class reflective journals 


Three variables related to student’s pre-class reflective 
journals 

3th 

4th 

7th 

9th 

11th 

Submission rate 

83.3% 

89.2% 

84.2% 

88.3% 

84.2% 

Share understanding 

54% 

69.2% 

78.2% 

79.2% 

76.2% 

Share discovery 

17% 

28.9% 

46.5% 

49.1% 

52.5% 


In the five reflective journals selected for grading, all the students revised their pre-class reflective journals 
after class. Some of them revised or extended their explanations on former understanding, and some of them found 
the answers to the questions they asked in their pre-class reflective journals. 

Findings from the collaborative project work 

The researcher’s original design of collaborative project was to let the students explore and give 
presentations in class time. Considering the limited class time and the large class (120 students), the instructor and 
the researcher arrived at consensus to require the students to make online presentations on Blackboard discussion 
board and not occupy the class time. 
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The researcher collaborated with the instructor and two graduate students in biology on designing the 
collaborative project topics. These topics were all related with real-life situations, and challenging for the students to 
fulfill the requirement with the knowledge they learned from the course. After arriving at consensus with the 
instructor on the design of the topics and learning activity requirements, the researcher divided the students into 12 
groups (10 per group) according to the alphabetized sorting of their family name, and made sure there were both 
males and females in all groups. Then the researcher created a thread for each group in the discussion board on 
Blackboard. The researcher posted each group’s topic and group members’ names in the first post in each thread, 
and sent an email to the each group about their topics, peers, and project requirement. In the next day, the researcher 
made a presentation of the detailed requirements of the collaborative projects. After the presentation, the researcher 
and the instructor answered the students’ questions about the collaborative projects in Q & A. 

The students were required to collaborate with their peers on the assigned topics, and post their findings 
and thoughts on the thread the researcher created for each group on the discussion board. Each group of students 
were supposed to work as an expert panel on their group’s topics and should give an online demonstration of their 
findings for the other groups. Each group was required to summarize their findings with PowerPoint slides and 
upload them to their group’s thread on the discussion board before the date when their project work was due. In 
addition, each group were required to submit a project report that briefly introduced their findings and summarized 
their collaboration process, including each member’s role and contributions to the group work. 

In the two days after the group work was due, the students were required to view other group’s projects and 
pick up five topics they were interested in, and post at least one comment on each project, such as doubts, 
understandings, and suggestions. In addition, the students were encouraged to keep an eye on the questions other 
groups asked in their own group’s thread and answer them in a timely fashion. 

In order to promote the students’ collaboration, the researcher emphasized that all the students in the same 
group were supposed to share the same grade according to the quality of their group work. However, there was a 
peer assessment for the students to evaluate each of their group mate’s efforts devoted to the group work. If a 
student were to receive a poor grade in the peer assessment, he/she would get a lower grade than the other students 
in the same group. Before the collaborative project, all the students were informed that they should work hard to 
improve their group work in order to get a higher grade for their group, but if someone were to receive a poor peer 
assessment grade, his/her grade would be decreased. The students were also informed that each member of the group 
would have the opportunity to evaluate each teammate’s contribution to the group work. The peer assessment was 
released on Blackboard two days after the group work was due. 

The closed-ended questions in the anonymous survey collected the data about the students’ overall attitudes 
towards the collaborative projects, problem-solving skills, learning gains, and collaboration. The survey results 
showed that a majority of the students had a positive attitude towards the collaborative project learning activity. Of 
the 78 students who finished the survey, 90% agreed that they felt the collaborative projects were interesting and 
they were willing to work actively on the collaborative projects. Up to 85% stated that they learned lots of new 
knowledge related to real life through the project. However, students were not sure whether the collaborative project 
learning activity could improve their information searching abilities in future. Approximately 40% agreed that the 
collaborative project helped them learn to obtain information from a variety of sources. Only 27% of the students 
agreed that they could apply the knowledge they learned to solve other problems, while only 23% asserted that their 
ability to analyze information had been improved. 

Slightly more than 80% of the students expressed a positive attitude towards their collaboration process, 
and 75.6% stated that the other members had showed respect to them and their learning styles. However, 14% of the 
students complained that they did not feel comfortable when asking for help from their peers, while 23% chose the 
option “I feel uncomfortable sharing information with others”. Slightly more than 52% agreed that they benefited 
from other groups’ demonstrations. Additionally, 52% hold a positive attitude towards the peer-assessment, 
reflecting that they “liked the idea of evaluating others”. 

There was an open-ended question in the anonymous survey requiring the students to give a brief summary 
of their own contributions to their group works. Students’ responsibilities in their project work could be classified 
into leader, compiler, and common contributor. The leaders usually came up with an idea as the whole outline of the 
group work, specified the topic, and assigned each member tasks. They also coordinated the other group member’s 
works, such as contacting group members, organizing meetings, and helping to keep everyone up to date on their 
roles and responsibilities. The compiler’s responsibility was integrating the materials all the group members 
contributed into a whole piece of group work. One compiler described his/her work as “pulling all the sources 
together for the group”, and “fixing the PowerPoint slides for well-organized presentation”. In some groups, leaders 
also played the role of compiler. The common contributors fulfilled the tasks assigned to them by their group 
leaders, and some of them contributed some resources for the group work. In sum, all the students contributed and 
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worked hard in their respective groups. Most students worked well and almost no one was cited by others as having 
been lazy or uncooperative in contributing to the group work. Almost no student reported that any conflicts occurred 
during the collaborative project work. 

However, there was one incident disharmony in the collaborative project activities. The instructor received 
the emails from six students who claimed that they missed the announcement of the collaborative project 
requirements, and their group members had nearly finished all the work when they noticed that they would have 
missed the project assignments. A decision was made by the instructor to make the six students into another group to 
work on an extra project topic. They would have 20% credits deduction for penalty. 

Students’ suggestions for improvement 

The feedback option in reflective journals and the open-ended questions in the survey provided the students 
with an opportunity to offer their suggestions to improve the instructional design of this course in future. Most 
students suggested that they needed more detailed and clearer guidelines for both the reflective journals and 
collaborative projects. In addition, the students expressed their desire for more timely feedback from the reflective 
journal assignments. They also reported they needed more time to complete their collaborative projects. 

In the survey about the collaborative project learning activity, most students admitted that they felt excited 
to see some real life examples, with which they were familiar. The examples were about how genetics explain 
human nature. However, the students wished to have a more detailed activity guideline and grading policy that 
would allow them to do a better job in the collaborative project activity. A similar sentiment was expressed about 
the reflective journal activity. Some students, especially the students who planned to apply for medical school, 
desired more instructions to tell them how to achieve higher grades., because they cared so much on their grade and 
GPA. At the beginning of the semester, the guideline for the reflective journal learning activity was not clear enough 
and caused some students’ problems in completing finished their first reflective. Based on the students’ feedback, 
the researcher revised the guideline of the reflective journal activity, so the students would better understand what 
they were required to do. 

A few students’ suggested they need more time to finish their collaborative project. Students stated that 
they would have had more in depth thinking and done a better job, but the limited time made their work into a busy 
work. 

Students also suggested that more timely feedback should be provided to the students about their 
performance in the reflective journals. They also desired more timely answers to their questions about the learning 
activity requirements. 

Instructor’s perceptions 

As mentioned earlier, the instructor was concerned that he might need to devote more time for preparing 
and grading the student-centered learning activities. He also worried that the students could not do well in the 
collaborative project learning activity. He had limited knowledge of the field of education. These were the reasons 
that he was concerned that the flipped classroom model might not work well in his classroom. 

The same native language and cultural background made the communication between him and the 
researcher more convenient, but the age gap and the different cognitive styles formed by long-term subject learning 
and research caused some misunderstandings between them. For example, when rating the students’ reflective 
journals, the researcher advocated “personalization”, giving students different and personal feedback, but the 
instructor advocated “standardization”, giving the students’ same and standard feedback according to the grades they 
got. 

The instructor was astonished at the students’ active performance and creativity in the learning activities. 
His new awareness of the students’ thought process from the reflective journal learning activity dispelled his 
misconception about the student laziness in reading the textbook and poor understanding in genetics knowledge. 
Moreover, he was pleasantly surprised to see the students’ creativity and active performances in the collaborative 
project learning activity. He even admitted that “these young students can build a skyscraper in a day” (personal 
communication. May 6, 2012). He was very satisfied with the students’ active performance in group discussion and 
Q & A during the class time. His concerns about the possibility that the students might have been distracted by the 
student-centered learning activities never materialized. 

The schedule of this course was tight, so he was afraid that the more than 100 students’ collaborative 
project and group presentation would take too much class time. That’s why the format of the collaborative project 
was changed from face-to-face into online format. 

Keeping the balance between the students’ peer pressure and competition was another concern the 
instructor had about the collaborative project. When the instructor discussed with the researcher about the student’s 
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guideline of the collaborative project, he expressed his concern that it was difficult to manage the 120 students, and 
some students might share the grade without active contribution in their projects. When the researcher told him that 
peer pressure and peer assessment can facilitate students’ collaboration and contribution, he questioned how to avoid 
the students competing with each other. According to the instructor’s suggestion, the researcher highlighted in the 
guideline that all the students in the same group were supposed to get the same grade but the students who failed to 
contribute, when reported by 3 other group members, would get a lower grade. The peer assessment technique 
effectively facilitated the students’ collaboration by peer pressure and also avoided them forming a competitive 
relationship. 

Challenges 

Throughout the semester, the researcher felt that the biggest challenge was working as an instructional 
designer and teaching assistant in a genetics course, which was not in the researcher’s academic field. The 
researcher had no background in science, and only learned biology in high school. In addition, as a Chinese student, 
the researcher had the difficulty in understanding biological terminology. In order to overcome this obstacle, the 
researcher worked hard to read the textbook in order to understand the subject matter in the course. In addition, the 
researcher collaborated with two biology graduate students. They explained the knowledge to the researcher and 
helped the researcher to understand. When designing the collaborative projects, the researcher also consulted with 
the two biology students. 

A big challenge for both the instructor and the researcher was the lack of time. The researcher had limited 
time to learn the biology knowledge. In the student-centered learning activities, the feedback should include the 
detailed evaluation for students’ performance and suggestions for their improvement. What concerned the researcher 
most was that there was limited time to give students detailed feedback on this talk. The time was so limited that it 
was impossible for the researcher to read through all the students’ reflective journals and give them detailed and 
personalized feedback, and this was why the researcher accepted the instructor’s recommendations to give the 
students “standardized” feedback. In addition, the instructor and the researcher had limited time for face-to-face 
discussion about the design of learning activities and students’ evaluations. They communicated more via email, but 
it made the discussion superficial and less timely. 

Student management was a challenge for the researcher. It was difficult to manage a large class of more 
than 100 students. The Blackboard course management system helped a lot in announcing, grading, grouping, and 
communicating with the students. Improving the peer collaboration among students was also an approach to 
decrease the workload for the researcher. 

Moreover, both the instructor and the researcher were concerned about the cultural barrier between them 
and the students. Although the instructor had been teaching in this university for more than 10 years, he always 
expressed difficulty in understanding American students’ thoughts. The instructor and the researcher both found it 
was too difficult to probe into the young American students’ inner world. The researcher was always worried that 
the misunderstandings caused by cultural barrier would have negative influences on the students’ learning. The 
cultural issues are very complicated and should be explored in future studies. 

Discussion 

Overall, the flipped classroom was an efficient teaching and learning model to facilitate the students’ active 
learning, independent thinking, knowledge application, and collaborative learning in a large biology class. It was the 
first time for the instructor to introduce student-centered learning activities in his class, and also the first time for the 
researcher to work as an instructional designer and teaching assistant in a large undergraduate science course. 
Although the overall goal of improving students’ engagement had been achieved through flipped classroom 
activities, there were some limitations to this study and room for improvement. 

It’s a transitional flipped classroom course 

The organization of the genetics course in this study may not be what some scholar talked flipped 
classroom. According to Bergmann, Overmyer and Wilie (201 1), a flipped classroom is called “flipped” because 
what used to be classwork is done at home by learning the basic concepts via teacher-created videos, and what used 
to be home work is done in class through participating in student-centered learning activities. Students not only 
should have self-directed learning at home via videos, but also should finish short tests or reflective surveys to 
evaluate their learning (O’Neil, Kelly, & Bone, 2012). Most of the class time should be spent on active learning 
activities. 
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However, in this study, the students learned basic concepts at home, but via textbooks. There was an 
attempt to evaluate their understanding of knowledge before class, through writing reflective journals, but not tests. 
Moreover, the time allowed for the student-centered active learning activities was not a large part overall class time. 
The classroom work had not been completely changed to activity-based learning. Although the instructor added 
group discussion and Q & A to in-class time, the in-class activity was simple, and lecture still filled a major part of 
class time. The researcher initially designed a collaborative project model and planned to implement it in class, but 
because of the tight schedule and large class size, it was carried out after class in online format. However, one whole 
class was spent for announcing the requirement and Q & A in order to help the students get familiar with the new 
learning approach. Nevertheless, it was the beginning of “flipping” the classroom. Although there was room for 
improvement and completing the task of “flipping” the classroom, this instructional model was a step in the right 
direction for implementing a flipped classroom model. It had made the course more like a flipped classroom model 
than a traditional lecture approach. In this sense, the genetics course in this study was called “transitional flip”. 

“Flipping the classroom requires more than video” 

Textbook was used as the pre-class learning material in the genetics course in this study, not video lecture 
that was usually used as a typical pre-class learning material in literature. In many flipped classroom models, video 
is viewed as a necessary component in flipping the classroom, and always used in small classes (Demetry, 2010; 
Moravec, Williams, Aguilar-Roca, & O'Dowd, 2010), but this is not necessarily essential. In another study with 
undergraduates, Demetry (2010) found that watching videos as the pre-class learning experience can stimulate the 
students’ interest in spending outside class time learning the basic concepts. 

The flipped classroom is not a specific activity, but an innovative model of teaching and learning which 
utilizes educational technology and student-centered active learning activities to positively impact the learning 
environment (O’Neil et al., 2012). Makice (2011) argued that flipping the classroom was an ideology evolved into 
empowering students to consume information outside class and demonstrate understanding of concepts in various 
ways. He summarized that it would be efficient only if videos were a part of the plan to connect students’ learning 
with active and real-life experience. Any flipped classroom model should be adopted to meet the needs of the 
students, or the learning effectiveness may be even worse than that in the traditional learning model. 

As Miller (2012) suggested, the primary focus of flipping the classroom should be on increasing students’ 
engagement. Any use of technology and learning activity design must support the goals of increasing students’ 
engagement, teacher’s productivity and facilitation, and the support for innovative instruction, to allow flipped 
classroom to be effective and efficient. Any technology or media which can efficiently support students’ learning 
should be adopted, but not limited to video. In this study, reading the textbook was the pre-class learning activity 
and pre-class journal was adopted to measure students’ understanding of knowledge before coming to class. 
Although textbook reading may not stimulate students’ pre-class learning interest as well as video lectures, this 
could be examined by future research in which more multimedia learning materials will be designed for pre-class 
learning. 

The value of collaboration in instruction 

The researcher was more aware of the value of the collaboration with the instructor, students, and the two 
biology graduate student subject experts. This collaboration did decrease the workload for the researcher. 

In this study, the researcher had the knowledge background in instructional technology, but had no subject 
knowledge background in biology; while the instructor had rich subject knowledge in biology, and had taught this 
course for many years in the traditional lecture format, but he had paid limited attention to educational innovations. 
The success of the transition from traditional lecture teaching approach to the flipped classroom approach required 
the instructor and instructional designer to collaborate and take advantage of their individual expertise. Prior to the 
beginning of the course, the researcher explained the theory of the flipped classroom to the instructor and they 
agreed on the design of learning activities. During the course, the instructor and the researcher kept in contact via 
email, and had a face-to-face meeting every week. However, their communications focused more on the students’ 
management, and ignored the integration of their educational expertise and subject expertise. That also caused the 
researcher to feel anxious when designing the collaborative projects. 

The collaboration between the instructional designer and subject experts also played an important role in 
this study. When designing the collaborative project topics, the researcher only met the instructor only one time. 
Because of this, the researcher was referred to the two biology students for help. They gave some illustrations of the 
subject knowledge the researcher should use potentially in designing the project topics and provided some cues. The 
two biology students played the role of subject expert in this course. Although they didn’t participate in the 
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instruction directly, their expertise had an important influence on the quality of instruction. In future, if more 
student-centered learning activities added, the more collaboration on instruction would be needed. 

The communication with the students helped the researcher know more about the characteristics of 
American undergraduate students, and facilitated the researcher and the instructor in bridging the cultural barrier. 
The researcher sincerely encouraged the students to provide feedback on learning activity design in various format, 
such as email, face-to-face meeting, and the “feedback” part in reading journal. The researcher was the teaching 
assistant in students’ eyes, but also learned American culture from the undergraduate students. At the end of the 
course, the researcher even felt reluctant to leave these students. The researcher learned lots about the inner world 
and life of American undergraduates. What the researcher appreciated most was the students’ thanks. Based on the 
students’ feedback, the researcher appeared to be accepted by them, and was more popular than the instmctor 
because the researcher was closer to the students’ age. 

A limitation of this study was that the data analysis on the students’ reflective journals was subjective. Only 
the researcher analyzed the reflective journals. In future study, more raters should be used and inter-rater reliability 
should be calculated. 

In sum, more research and other types of studies need to be conducted with students in diverse disciplines, 
and diverse class organizations, to get a more comprehensive picture of the effectiveness of a flipped classroom 
model in different learning environments. More experiments are needed to complete the transformation from a 
traditional lecture -based instructional model to a flipped classroom instructional model. 
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Chronotypes and Online Learning: 
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Abstract 

Introducing a new theoretical framework of chronobiology, this paper presents a survey study examining 
learners’ choices, participation and performance in two discussion-heavy online history courses. The study 
comprises two major parts: a repetition study and an exploratory study. Adopting the research methodology 
developed by the same researchers in a previous study, the repetition study confirms the statistically significant 
relationship among the three factors for choosing online learning: perceived level of control, independence and 
satisfaction; as well as between learners’ chronotypes and their preferred online learning time. The second part of the 
research explored the level of participation and performance of learners having different chronotypes in two 
discussion-heavy online courses. The results indicate there was a statistically significant relationship between 
students’ chronotypes and the time of their activity in online courses. The overall level of student participation 
affected student performance in online course— students who were more active in LMS were likely to have higher 
final course grades. Besides, students who were more active in group-wiki were likely to have higher group wiki 
grades. Implications and limitations of the study are discussed. 

Introduction 

Online learning is becoming more and more affordable, accessible and suitable for today’s learners who can 
enjoy the convenience and efficiency of learning while staying connected all the time. While “the world is flat”, 
classrooms are becoming flat and borderless thanks to the development of online learning technology. “Online 
enrollments have continued to grow at rates far in excess of the total higher education student population, with the 
most recent data demonstrating continued substantial growth (Allen & Seaman, 2010, p. 1). Previous studies have 
identified some personal factors affecting online learner participation and patterns of participation such as learners’ 
background knowledge (Ross, 1996) and content area experience (Vonderwell & Zachariah, 2005) as well as 
instructional factors such as instructional tasks (Vonderwell & Zachariah, 2005) and instructor interventions 
(Vrasidas & Mclsaac, 1999). Learners’ individual differences, such as mental abilities, cognitive controls, cognitive 
styles, learning style, study habits, personality, communication skills, and prior knowledge and experience would 
affect online learners' learning outcomes (Picciano, 2002). 

Therefore, educators need to give careful consideration to individual differences to provide online learners 
a more effective learning environment (Halim, Ali & Yahaya, 2011). This research hopes to examine online 
students’ choice, participation and performance in online learning from the perspective of one particular personal 
factor of learners — chronotypes. Human beings with different chronotypes would function differently at different 
times of a day, which is determined by their inner biological clock. Morning-types or “larks” prefer to wake up early 
in the morning while can hardly stay up beyond their usual bedtime; however, evening-types or “owls” show a 
preference for sleeping at late hours at night and often have difficulty in getting up in the early morning (Giannotti, 
Cortesi, Sebastiani, & Ottaviano, 2002). With regard to online learning, Luo, Pan, Choi, Mellish & Strobel (2012)’s 
research studied the effect of chronotypes on university students’ choice of online learning and found out that there 
is a correlation between chronotypes and students’ preferred time to do online learning. The study pointed out that 
online learning offers students a viable option to choose a time to attend or work on course in terms of their 
circadian preferences. 

This research thus hopes to make a contribution and to add to what is already known about online learning 
from the perspective of chronotypes, one of learners’ most indispensable biological functions, by examining the 
learners’ choice, participation and performance in two online discussion-heavy courses. The purpose of this research 
is to shed light on the understanding of learners’ participation level and performance in online learning taking into 
consideration students’ chronotypes, and ultimately benefit the design and implementation of online instruction. 

The research comprised two major parts: a repetition study and an exploratory study. Using the research 
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methodology developed by the same researchers in a previous study, the repetition study aimed to investigate the 
relationship among the three factors for choosing online learning: perceived level of control, independence and 
satisfaction; as well as the relationship between learners’ chronotypes and their preferred online learning time. The 
second part of research explored the patterns of learners’ participation in two discussion-heavy online courses using 
the log-in data in Black Board Learn, the learning management system (LMS) in use at the institution of the study. 

In this study, we used the log-in data of students’ hits on the online courses as a whole as well as their hits on two 
important asynchronous instructional components in Blackboard Learn: discussion boards and group wiki. We tied 
the log-in data then to learners’ chronotypes and performance in the online courses for further analyses. 

The research questions were: 

1. Are there any variations in students’ level of activity in LMS at different time period of the day and final course 
grade by chronotype? In addition, is there any correlation between students’ level of activity in LMS and final 
course grade? 

2. Are there any variations in students’ level of activity in discussion board by chronotype? In addition, is there 
any correlation between students’ level of activity in discussion board and final discussion board grade? 

3. Are there any variations in students’ level of activity in group wiki by chronotype? In addition, is there any 
correlation between students’ level of activity in group wiki and final group wiki grade? 

Literature Review 


Time Issues in Online Learning 

In terms of the time issue in online learning, students perceive that online learning has several advantages 
over traditional methodologies: saving them commuting time; fitting in better with their schedules; and enabling 
them to take more courses (Dutton, Dutton & Perry, 2002). Instmctional activities emphasizing convenience and 
time management are preferred by online students (Butler & Pinto-Zipp, 2006). Poole (2000) studied students’ 
participation in an online discussion-oriented course and found out that students participated in online discussions at 
times most convenient to them: there were more posts made on Saturdays than any other day of the week. The study 
also indicated that as most of the participants were teaching during daytime hours on weekdays, fewer posts were 
made during daytime than evening hours throughout the week. Indeed chronotypes (inner biological clock) is 
relevant to education. The circadian system cannot adapt to changes in sleep/wake schedules fast so that it has 
consequence on education: adolescent students who have vastly different sleep/wake schedules between school 
nights and non-school nights such as weekends or holidays would find it hard to reconcile these schedules and 
therefore experience fatigue, difficulty in falling asleep at night and awakening in the morning when school days 
resume (Carskadon, Acebo & Jenni, 2004; Battro, Fischer & Lena, 2008). 

From late 1970’s, there has been a number of the research on the relationship between the instructional 
conditions in the classrooms and the effect of time preferences on students. In these studies the significant 
relationship(s) between learning style, time preference, assigned instructional schedule, and achievement in certain 
student population have been identified (Dunn, Dunn, Primavera, Sinatra & Virostko, 1987). In terms of online 
learning, Luo et al. (2012) studied the effect of students’ circadian preferences on online learning and demonstrated 
that it is very likely that learners would choose a time to engage in online learning according to their circadian 
preferences as a way to minimize their “social jetlag” (Jovanovski & Bassili, 2007), which refers to the 
misalignment of the biological and social clocks, which cause people to function daily. 

Discussion Boards and Group Wiki in Learning Management System 

Online discussions form a constructivist learning environment for participants to interact and collaborate 
(Rovai, 2007; Cheung, Hew & Ng, 2008). The constructivist approach of online discussions encourages learners to 
investigate multiple perspectives on problems or ideas (McLoughlin & Mynard, 2009). 

Discussion boards are often text-based and are integrated with an online learning management system such 
as Blackboard or WebCT. In contrast to synchronous online discussions which require real-time access and 
interaction among instructors and learners, discussion boards are asynchronous so that it enables users to 
communicate through it at different times (Weisskirch & Milburn, 2003). Instructors or learners may start new 
threaded discussion topics and then others can response to the topics by posting messages, which are displayed in a 
threaded format with fields such as authors, topics and posting time (Tu, Blocher & Gallagher, 2010). According to 
Vonderwell, Liang & Alderman, (2007), asynchronous discussion boards could provide learners with independence 
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in learning regardless of time constraints and enable them to monitor their own learning process through instructor 
and fellow learners’ feedback. 

Chao (2007) defined wiki as “a website containing a collection of web pages that can be edited by visitors 
to the site (p.2)”. Wiki allows people to collaboratively create and edit the web contents (Chao, 2007). With the 
ability of communication and knowledge building feature, wiki is commonly used as a tool for collaborative 
learning in education (Boulos, Maramba, & Wheeler, 2006; Engstrom & Jewett, 2005; Parker & Chao, 2007; 

Wagner, 2004). Researchers suggested several ways to use wiki in the classroom and effectiveness of the tool was 
studied (Parker & Chao, 2007; Elgort, Smith, & Toland, 2008; Witney & Smallbone, 2011). The study ofElgort et 
al. (2008) showed that wiki encouraged students to participate better. Students mentioned that wiki was a good tool 
for collecting and organizing information. Furthermore, instructors expressed that wiki made the management and 
student tracking easier (Elgort et al., 2008). Witney and Smallbone (2011) also mentioned that wikis do help the 
student experience of collaborative learning, but emphasized that adequate support and scaffolding is recommended 
as well. Overall, researchers introduced wiki as a common tool for collaborative learning in online courses 

Online Learners’ Participation and Performance 

A literature review study done by Hrastinski (2008) indicated six levels to conceptualize learner 
participation in previous study: 1) online participation equals to the number of times a learner access an online 
course; 2) online participation equals to the messages or words a learner write in an online course writing; 3) online 
participation equals to writing contributions that are of high quality; 4) online participation equals to the number of 
writings written and messages read by a learner in an online course; 5) online participation equals to the number of 
messages that are perceived as important written by a learner in an online course: 6) online participation is related to 
learners’ taking part in a rewarding dialogue. 

Mason (1994) categorized online learners into three groups based on in their online participation: active 
participants, lurkers who read messages but do not post, and those who do not take part. Similarly Taylor (2002) 
used three distinct groups to show participation patterns in online discussions: workers, proactive participation group; 
lurkers, peripheral participation group; and shirkers, parsimonious participation group. Workers actively participate 
in the discussions and visit the class website regularly. Lurkers participate occasionally and often in a “read-only 
mode.” Shirkers perform the minimum required tasks with fewer postings and visits to the class website. 

Student performance outcomes do not have a definite measurement as it relies on the course content and the 
nature of students. Successful completion of a course, course withdraws, grades, knowledge and skills gained are 
some indicators of students’ performance, which can be measured through multiple ways such as testing, written 
assignments, individual projects, group projects and so on (Picciano, 2002). In terms of the previous studies on 
predicting the success of distance learning, they largely focus on the demographic (age, gender, marital status, 
occupation, etc.) and personality traits of the learners (Navarro & Shoemaker, 2000). 

Research Methodology 

Acting as both a repetition and an exploratory research, this research adopts the survey in a previous 
research conducted by the same authors, and adds 5 additional questions asking participants’ preference for online 
learning. The validated survey contains three sections: a demographic section, an online learning section and a 
chronobiology section. The online learning section contains questions such as participants’ preferred time of 
conducting online learning; participants’ preferences of different kinds of online instructional activities and why do 
participants choose online learning. The chronobiology component is based on the Munich ChronoType 
Questionnaire (MCTQ) (Roenneberg, Wirz- Justice & Merrow, 2003), which is a validated instrument to assess the 
chronotypes of participants. The research procedure is demonstrated in Figure 1 below. 
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Part 1 : A repetition study 

-Calculating MSFSC to determine learners' chrono types 

-Correlation analyses between three factors for choosing online 
learning: perceived level of control, independence and satisfaction; 
learners 7 chrono types and their preferred online learning time 

V 


Part 2 : An exploratory study 

-Kruskal-Wallis test/ANOVA analyses on student 
participation level in online courses, taking into 
consideration the chronotypes 

—Correlation analyses between chronotypes and student 
performance 


—Correlation analyses between student participantion level 
and performance 


Figure 1. The research procudure of this research. 


"\ 


■"N 


Data Collection and Analysis 

In the repetition study, the survey was distributed online to full-time residential undergraduate students 
registered in two core curriculum online courses at a Midwestern university: HIST104 and HIST152. These two 
online courses were delivered through Blackboard Learn in the 2012 fall semester. 153 students in total received the 
online link to the survey, with a return of 88 valid responses. Among these responses, 28 were from male 
participants and 60 were from female participants. 

In the exploratory study, the students’ participation data in the 2012 fall semester collected were as follows: 
the timestamps recording each participant’s hits on the Blackboard Learn courses as a whole; each participant’s hits 
on the specific instructional components of discussion board and group wiki; and the sum of hits in the Blackboard 
Learn tracking database for each student that took place during the listed hour of the listed day. The participation 
data were obtained with the help from the Office of the Vice President for Information Technology at the 
Midwestern University. The performance data were students’ final discussion board and group wiki grades, which 
were downloaded from the Blackboard Learn gradebook with the help of the instructor. 

The survey data and participation data were analyzed with the help of Microsoft Excel ® and SAS ®. In 
terms of the survey data which needed additional coding, such as category of the participants’ preferred online 
learning time, two researchers first did the coding separately and then compared each other’s coding to reach an 
agreement, in order to ensure reliability of data analysis. 

Validation of Chronotype Instrument 

As there are two means of obtaining participants’ chronotypes: research participants’ self-assessment 
(7-point scale) and a calculated chronotype (MSFSC) based on the participants’ responses to the standardized 
questionnaire, we validated the chronobiological instrument again according to the same approach as in previous 
studies - MSFSC is the mid-sleep on free days corrected for differences between sleep duration on free days and 
sleep duration on work days (Roenneberg, Kuehnle, Pramstaller, Ricken, Flavel, Guth & Merrow, 2004) and is used 
for determining chronotype (Ktihnle, 2006). Similar to the previous study, Table 1 below indicates participants’ 
self-assessment and MSFSC were highly correlated (Pearson correlation coefficient was 0.51 with P value < .0001). 
Thus MSFSC was adopted for further analysis. 


123 


Table 1 


Correlation analysis between MSFSC and the self-evaluated chronotype Correlations 


Pearson Correlation Coefficients, N=88, Prob>|r| under H0:Rho=0 


MSFSC calculated by MCTQ 
questionnaire 

Self-assessed chronotype 

MSFSC calculated by MCTQ 

1 

0.50662 

questionnaire 


<.0001* 

Self-assessed chronotype 

0.50662 

<.0001* 

1 

Note. *p<.01 




Results 


Results of the Repetition Study. 

In terms of the distribution of learners’ chronotypes, which are calculated by MSFSC— the mid-sleep on free 
days corrected for differences between sleep duration on free days and sleep duration on work days (Kiihnle, 2006), 
this research confirmed the findings of the authors’ previous research. As Figure 2 demonstrates, the MSFSC 
distribution of the participants was close to a normal distribution (p=6.93, o=1.56). Figure 3 exhibits the distribution 
of participants’ time preferences for doing online learning (morning=l, morning-afternoon=2, afternoon=3, 
afternoon-evening=4, evening=5, evening-morning=6, split=7 and all= 8). Similar to the previous study, the 
distribution was right-skewed (p=4.92, o=1.91), implying most of the participants prefer to do online learning at a 
later time of a day. 

Distribution of Chronotype as Measured by MSFSC 

The UNIVARIATE Procedure 


Distribution of MSFSC 



Figure 2. Raw MSFSC distribution of the participants. 
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Distribution of Preferred Online Learning Time 

The UNIVARIATE Procedure 


Distribution of Prefer 



Curves Normal(Mu=4.9205 Sigma=1.9132) Kernel(c=0.79) 


Figure 3. The distribution of participants’ time preferences for doing online learning. 


Results in Table 2 exhibit that the three existing factors of learners’ choice of online learning — perceived 
level of control, independence and satisfaction were highly correlated with each other (with all p-values <. 01), 
which was in accordance with the previous study as well. 


Table 2 


Correlation analysis between three factors of choice of online learning 


Pearson Correlation Coefficients, N= 

=88, Prob>|r| under H0:Rho=0 



Level of control 

Level of independence 

Level of 




satisfaction 

Level of control 

1 

0.39087 

0.69408 



0.0002* 

<.0001* 

Level of independence 

0.39087 

1 

0.57606 


0.0002* 


<.0001* 

Level of satisfaction 

0.69408 

0.57606 

1 


<.0001* 

<.0001* 


Note. */?<.01 
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In addition, the results further confirmed the statistical significance between online students’ chronotypes 
and their preferred online learning time. Table 3 shows the correlation between MSFSC and participant’s time 
preference was quite strong (P value=0.01), which indicates an early-type online learner might prefer studying 
online in the morning while a late-type might prefer studying at a later time of the day. 


Table 3 

Correlation analysis between chronotype and choice of online learning time 

Pearson Correlation Coefficients, N=88, Prob>|r| under H0:Rho=0 



MSFSC calculated by MCTQ 

Students' preferred online learning 


questionnaire 

time 

MSFSC calculated by MCTQ 

1 

0.27107 

questionnaire 


0.0106* 

Students' preferred online 

0.27107 

1 

learning time 

0.0106* 


Note. *p<.01 


Results of the Exploratory Study 

In order to explore if there are any variations in students’ level of activity in LMS at different time of the 
day by chronotype. we first calculated each student’s total number of hits in LMS at different time period of the day 
during the semester: morning (7am -12pm), afternoon (lpm-6pm), evening (7pm-12am), and midnight (lam-6am). 
We then performed Kruskal-Wallis test/ANOVA test on the number of hits in each time period as well as the total 
number of hits, using chronotype as the categorical predictor. The selection of test depends on the normality of the 
dataset. We used Kruskal-Wallis test when the normality assumption was violated. Otherwise we used ANOVA test. 

Table 4 Test on LMS activity 



Test 

P value 

Morning 

Kruskal-Wallis 

0.0178* 

Afternoon 

ANOVA 

0.7318 

Evening 

Kruskal-Wallis 

0.0315* 

Midnight 

Kruskal-Wallis 

0.1418 

Total number of hits 

Kruskal-Wallis 

0.4139 


The result in table 4 indicates there was a significant difference in students’ level of activity in the morning 
and evening by chronotype as the p values for both tests were less than 0.05. Therefore we further examined the 
Wilcoxon mean scores obtained from the tests on morning and evening data (table 5 and table 6). From the tables we 
can see that the morning-afternoon (type 2) and the morning (type 1) had more hits and therefore were more active 
in LMS in the morning, while in the evening, the all (type 8) and the evening (type 5) were the most active groups. It 
was also noticed there was no significant difference in the total number of hits among the participants’ of different 
chronotypes. 
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Table 5 Wilcoxon mean scores from analysis of morning data classified by chronotype 


Chronotype 

Wilcoxon mean scores 

2 

67.7 

1 

65.8 

8 

54.0 

3 

49.2 

7 

47.4 

5 

44.5 

4 

41.4 

6 

26.7 


Table 6 Wilcoxon mean scores from analysis of evening data classified by chronotype 


Chronotype 

Wilcoxon mean scores 

8 

59.7 

5 

59.4 

7 

46.0 

4 

45.7 

6 

45.4 

3 

31.6 

1 

29.4 

2 

8.3 


In order to determine if there are any variations in students’ final course grade by chronotype, we 
performed Kruskal-Wallis test on the final course grade using chronotype as the predictor. The result indicates there 
was no significant difference in students’ final course grade by chronotype fp — 0.0752). We then further explored 
whether there was any correlation between students’ level of activity in LMS and final course grade using the 
Spearman’s rank correlation analysis (due to the non-normality distribution of the data) and the result showed they 
were positively correlated (correlation coefficient= 0.340, p=0.0012), meaning students who are more active in LMS 
are likely to have higher final course grades. 

To determine whether there are any variations in students’ level of activity in discussion board by 
chronotype, we performed Kruskal-Wallis test on the total number of hits in discussion board, using chronotype as 
the predictor. The result showed no significant difference (p=0.5042). Next, we explored whether there was any 
correlation between students’ level of activity in discussion board and final discussion board grade using 
Spearman’s rank correlation analysis and the result showed no significant correlation (correlation coefficient^ 0.209, 
p=0.0509). 

To determine whether there are any variations in students’ level of activity in group wiki by chronotype, we 
performed AN OVA test on the total number of hits in wiki, using chronotype as the predictor. The result showed no 
significant difference (p — 0.6 1 1 9). Next, we explored whether there was any correlation between students’ level of 
activity in group-wiki and final group wiki grade using Spearman’s rank correlation analysis and the result showed a 
positive correlation (correlation coefficient^ 0.470, p<0.0001), meaning students who are more active in group wiki 
are likely to have higher group wiki grades. 


127 



Discussion and Conclusion 


Kiihnle (2006) defined 2.5% at each end of the MSFSC distribution as extreme chronotypes, with extreme 
late types (MSFSC>7.25) and extreme early types (MSFSC<2.17). When comparing the distribution of MSFSC in 
this research to that in Kiihnle, it is noticed in Figure 4 that there tend to be many more extreme late types (37.5%) 
and fewer extreme early types (1.14%) among online learners as compared to the extreme types among the general 
population. 



Distribution of Chronotype as Measured by MSFSC 
The UNIVARIATE Procedure 


Distribution of MSFSC 



MSFSC 

- Normal(Mu=6.9295 Sigma=1.556) 


MSFSC distribution of general population ( Kiihnle , 2006) MSFSC distribution of the online learners 

participating in this research 

Figure 4. A comparison between MSFSC distribution of general population (Kiihnle, 2006) and MSFSC distribution 
of the online learners participating in this research. 


Furthermore, undergraduate online learners with different chronotypes would have their different preferred 
online learning time. More specifically, their self-reported preferences were corroborated by the Blackboard Learn 
log-in data. The early-bird students, which include the morning-afternoon and the morning types, are more active in 
the Blackboard Learn in the morning. On the other hand, the all type and evening type students are shown to have 
the most activity in Blackboard Learn in the evening. This results show that students are experiencing learning 
benefit from the advantage of online course, which is ‘the flexibility of time’. 

However, in spite of their preferences and different level of activity in the morning and in the evening, the 
undergraduate students’ overall performances in the online courses are not significantly different. Such finding is in 
contradiction to the previous research indicating the effects of time effects on student achievement in traditional 
classrooms, for instance, Randier & Freeh (2006)’s research implying that morning-type college students would 
perform better than their evening-type counterparts. To this end, online learning may outperform traditional 
face-to-face learning in terms of its convenience to learners and especially the learners who experience time issues 
or conflicting schedules in real life. In other words, that we didn’t find significant difficulties in students overall 
performance based on chronotype, let’s us cautiously argue that the offering of online courses mitigated any effect of 
chronobiology previously reported for face-to-face classes. 

This research also found that students with different chronotypes access the asynchronous instructional 
components in Blackboard Learn, the discussion boards and the group wiki with a similar frequency and that 
chronotype was not a significant variable. It is the overall level of activity that affects the students’ final grades. This 
finding confirms Davies & Graff (2005)’s research, which argued the level of interaction in Blackboard would have 
effect on the grades of undergraduate students. Overall, our research showed the benefit of online learning by 
showing the correlation among students’ chronotype’, their preferred time to study in LMS, and participation level. 
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Although the study showed no significance between different chronotypes and performance level, the research 
furthermore leaves the room for deeper investigation; will the same student have lower performance level if she/he 
does not follow their own chronotype - say in a face-to-face course that is misaligned with the preferred time? 
Research could also be done on the relationship among chronotypes, student participation level in synchronous 
online instructional components (such as synchronous online discussion, virtual office hours), and student 
performance. In addition, researchers could investigate the satisfaction level among the students of different 
chronotypes. Since most traditional schools are scheduled for morning/afternoon person, evening chronotype or very 
early morning chronotype may have more satisfaction level to the online course. These potential research areas 
could support the reason to choose online over face-to-face classroom. 

Limitation for this research is that sample size is confined to two history classrooms with certain activities. 
A course with other subject and grades distribution, and more students could possibly show more variance for 
participation level and performance level. 

This research is significant to researchers, instructors, and instructional designers who are curious about the 
effectiveness of online course. With the other researchers of online learning, this research could be a good reference 
for pursuing why online learning should be delivered. 
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Introduction 

Colleges and universities have rapidly migrated classes to the online arena. “An estimated 4.6 million 
students took at least one online course in fall 2008” (Mortland, 2007). Barriers to engaging and teaching students 
have been identified since the beginning of distance education. In order to assure quality in online courses, Big 
Sandy Community and Technical College has recently introduced a Quality Assurance initiative in order to diminish 
the previously identified barriers for online student success. Big Sandy Community and Technical College is a 
member of the Kentucky Community and Technical College System and is located in a rural community in eastern 
Kentucky. 

The initiative began as a result of a statewide committee for distance learning’s suggestions for good 
practice. A rubric was developed to use as a check list for class creation and validation. The college enlisted twelve 
distance learning veterans to act as mentors and reviewers in this initiative. Each distance learning course must be 
reviewed before it is put on the college course schedule. The initial push for courses to be reviewed began in fall of 
2012 and continued into spring of 2013. The students that participated in this study were randomly selected from 
registered students for summer 2013. The purpose of the study was to evaluate the student and faculty perceptions of 
the barriers and the effect of the Quality Assurance process had on courses being taught online. 

Literature Review 

As a response to the migration of classes to the online environment it is important to identify all of the 
variables that can stifle the success of the students. At different intervals since the inception of online learning there 
have been several identified perceived barriers to success and engagement. Some barriers seem to be constant while 
others have evolved with technology over time. This review identifies the perceived barriers in online courses over 
the last twelve years. 

Flexibility has been identified as the initial reason for most students to try classes in the online format and 
the courses were initially designed in a way which was student centered (Institute for Higher Education Policy, 

1999). In the beginning of the online education era some of the barriers for success and engagement were identified 
as the inability to use technology and the lack of face to face contact between students and the instructor (Perreault, 
2002, p 3 14). Another factor that was identified was the lack of research of effective methods for online teaching 
and learning. The goal of Perreault et al (2002) was to seek effective methods of teaching and learning in order to 
publish the findings to share with the distance education community. At this time faculty support ranged from none 
to having mentors as well as graduate assistants to aid in developing courses. The current trend is to have 
instructional designers on staff to help the faculty members create courses that are effective for teaching and 
learning. 

Instructional barriers were identified again in 2006 to include needed professional development by 
instructors, extensive time demands in converting traditional courses to online courses and no clear way to teach 
kinesthetic materials online (Zirkel, 2006). Some of the identified student barriers also included was isolation, 
computer literacy, interactivity between students and instructors and poor time management skills. 

In a study that was completed a year later, student perceived barriers were identified as isolation and no 
sense of community involvement (Xiaojing, 2007). Yet another study concluded that the latest barriers from the 
faculty perspective was been identified such as the inability to use technology, lack of support from administration 
and difficulty in learning to use tools to create engaging content (Hannum, 2010, p 74). Some of the barriers over 
time have remained constant while others have dissipated. 


131 



Hypothesis 


The hypothesis of the research is that the Quality Assurance initiative will diminished some of the 
previously identified barriers that deter learning in online courses. New methods of engagement are being used in 
online education. The Quality Assurance rubric has built in ways to scaffold and engage students. Communication 
strategies between students and with the instructor are embedded within the rubric. The usage of course management 
systems which include forms of asynchronous and synchronous communication is required. The evolution of 
distance learning has changed the barriers to create more successful online educational experiences. 

The guiding questions of the research are: 

What will students identify as barriers to success? 

What affect did the Quality Assurance rubric have on success rates of students? 

What is the faculty’s perception of the barriers to success for both faculty and students? 

What are the most engaging activities for online courses? 

Methodology 

The methodology used in this study was a mixed method. To investigate the barriers perceived by students 
and faculty two surveys were used that included both closed and open-ended questions. The student survey consisted 
of 20 questions while the faculty survey was made up of 14 questions. The first survey was sent via email to 138 
students that were randomly selected from the BSCTC summer schedule. There were 35 student responses to the 
survey. The faculty survey was sent via email to 128 full-time faculty members. Only 36 faculty members responded 
to the survey. The surveys were created as a result of the findings of previous research in the literature review and 
the researchers experience in distance education. 

Quality Assurance Rubric 

The Quality Assurance Rubric (QA rubric) consists of six major areas of review. These areas include: 
course introduction and navigation, course design and organization, course and learning outcomes, instructional 
design and delivery, technical support with resources and compliance, and finally assessment with evaluation and 
feedback. The college has also identified a simplified course template in order for all of the online courses to have 
the same menu items. The goal was to make navigation easier for students to go from course to course. The course 
management system that is currently being used is Blackboard. 

The rubric is used by QA mentors that evaluate and work with faculty members on each course being 
taught online at BSCTC. The QA mentors are also members of the local Distance Learning Team at the college. 
These members have been identified as leaders in dealing with Blackboard and teaching online. Each section of the 
rubric has four to six areas of concern. The courses are carefully evaluated and each item is given a yes, no, or needs 
work. If the reviewer deems that the course needs more work, then the mentor will help the faculty member get the 
course up to college standards before the course is put on the college schedule. 

Some of the items that are covered in the QA rubric include promoting interaction and ways to 
communicate with peers and instructor, activities to enhance student learning and involvement with critical thinking 
and problem solving as well as strategies of awareness of current progress. Other areas that are covered include 
support information, ADA compliance, correct usage of copyright laws, and the usage of Early Warning system or a 
similar process to alert students about poor performance. Also the rubric covers items such as posting class/course 
expectations, logical organization of course content and grade center, learner engagement through instructional 
strategies and the usage of at least three technologies throughout the course. The technologies can include but are 
not limited to multimedia events such as videos, visual presentations, audio files, Softchalk exercises and other 
learning technologies. Initially there was resistance to the rubric implementation. Several faculty members felt that it 
diminished their academic freedom. 


Faculty Member Findings 

The online teaching experience of the faculty participants were that 23 had over 10 years of experience 
while 7 members had between 5 and 10 years of teaching experience and only 5 had between 2 and 5 years of 
experience. The faculty member participants were 49% female and 51% male. The participant’s ages ranged from 
30-60+ years. The number of male and female participants was approximately equal with 48.6% male and 51.4% 
female. 
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Faculty Member Demographics 

Online Teaching Experience 

< 1 year 

11.4% 

between 2-5 years 

28.6% 

between 5-10 years 

48.6% 

over 1 0 years 

11.4% 


Gender 

Male 

48.6% 

Female 

51.4% 


Age 

21-29 

0.0% 

30-39 

22.9% 

40-49 

31.4% 

50-59 

20.0% 

60 or older 

25.7% 


When faculty members were asked the open-ended question of what their most engaging activity included 
in their online courses, a wide range of responses were given. The most prominent response was discussion boards. 

Almost 37% of the faculty members identified discussion boards as their most engaging activity. Other 
activities that were prominently identified included research papers, blogs, Facebook interaction and self-reflection 
exercises. 

The faculty members were asked if the QA rubric initiative had affected their success rates 17% of the 
faculty members identified that it had positively affected them. Another 42.9% of the faculty members stated that 
they didn’t really know how it had affected their success rates and 40% answered that it had not affected their 
success rates. 62.9% of the faculty deemed that they need more tools (and knowledge to use them) to create more 
engaging content for online courses. In another survey question, faculty members were asked if they would like to 
attend professional development activities that include strategies for engaging online students and 82.9% said yes. 

When asked if faculty were supported by administration in the creation of online courses the response was 
yes for 82% of the participants. Of the 36 faculty participants, 91% answered that their course was engaging to 
students. The instructors were asked to identify in and open-ended question the perceived barriers for their success 
as a teacher and the responses were varied. The most prominent responses were lack of time, tools, software and 
professional development to create engaging materials. Lack of face-to-face contact with students was also 
mentioned several times. The bulk of the rest of the responses included ideas that contained deficits of the student 
such as lack of time management skills and interest. 

In another question faculty were asked to identify possible barriers for success of the students. There was a 
myriad of responses to this question. The responses included items such as technology, lack of self-discipline, time 
management skills, motivation and knowledge of taking online courses, while several faculty members identified no 
barriers to success for students. 

Student Findings 

The findings for the student’s surveys were very different than the answers of the faculty members. The 
survey respondents were 76% female and 24% male. The student ages ranged from 18 to 64 with the largest 
percentage being in the 18-24 range. 
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Student Demographics 

Ethnicity 


Caucasian 

91.4% 

Black or African American 

5.7% 

Asian 

0.0% 

Native Hawaiian or Other Pacific Islander 

0.0% 

American Indian or Alaska Native 

0.0% 

Other 

2.9% 


Age 


18 to 24 

37.10% 

25 to 34 

28.60% 

35 to 44 

17.10% 

45 to 54 

8.60% 

55 to 64 

8.60% 

65 to 74 

0.00% 

75 or older 

0.00% 


Gender 


Female 

74.30% 

Male 

25.70% 


The students were asked if they had ever felt isolated within an online course and 37.1% of the students 
responded yes. When asked about communication mediums used in their online courses 100% of the students 
identified email. Students also identified discussion boards and blogs as being used. In another response, 61.8% of 
the students acknowledged that their instructor had created a community environment in their summer online course. 

Students were asked to identify barriers to success in online courses and only 1 8 of the students responded. 
The most common response was lack communication and/or feedback from instructors. Several students identified 
the lack of face-to-face contact with other students and the instructor. Also, there were 6 students that stated there 
were no barriers to success for students in online courses. 
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How did you communicate with your instructor in your summer 2013 course? 
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In an open-ended question format students were asked to pinpoint the most engaging activities in which 
they have participated in their online courses. A wide range of answers were given with discussion boards being the 
most prominent. Other responses such as MyLabs and Leam Smart were given as well as team projects, computers 
and assignments. 83.9% of the students responded that they were motivated by course materials and design in their 
summer online course. 

In the last questions the students were asked if they had taken courses before the QA initiative had been in 
effect and only 22 students had a prior online course. Of these 22 students, only 6 of them stated that their summer 
course experience better than previous courses. Five students stated that the courses were not better than previous 
courses and the rest of the students either didn’t answer the question or didn’t take courses before the QA initiative 
began. 


Conclusions 

The faculty perceived barriers to their success have remained constant throughout the life span of distance 
learning. It seems that faculty members are constantly striving to learn strategies and technologies that will make 
more engaging courses. Professional development is needed to keep up with cutting edge technologies and how to 
use them in education. The perceived barriers of students that were identified by faculty have changed somewhat. 
The burden of responsibility is seen to be on the student instead of the course or faculty member. Courses are now 
more student centered. 

The students pinpointed barriers to success that did not match the barriers found in the previous research. 
The biggest concern of the students was lack of communication with the instructor. The students put the brunt of the 
barriers to success on the instructors. Faculty members and students as a whole would like the face-to-face contact 
that comes with traditional courses. Both faculty and students perceive discussion boards as a highly engaging 
activity in online courses. 

There is enough evidence from this research to determine that the Quality Assurance rubric has been 
successful in the endeavor to affect quality in online courses. A group of students and faculty did recognize the 
benefits of the changes in the courses. Students have also inferred that they have experienced engaging courses. 
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Abstract 

In light of an anemic economy and looming additional educational budget cuts, it is critical for educators to stay 
abreast of new technologies that will aid students in STEM disciplines. The purpose of this presentation is to share 
effective teacher education strategies that utilize constructivism and technology to increase skill level and content 
knowledge in elementary mathematics and science - through the implementation of source ware and free Web 
hosting and Web 2.0 tools. 


Introduction 

In light of globalization, mediocre international and national tests scores (Mathis, 2011; NCES, 2011; 2012) and 
a recessive economy, educational pundits and instructors must seek “free” new and emerging technologies that will 
aid students in the Science, Technology, Engineering, and Mathematics (STEM) disciplines. Concurrent research 
shows the strong linkages between motivation and learning (Bandura, 1991; Eccles, Wigfield, & Schiefele, 1998) 
when using the constructivist approach (Gagne, 1985; Papert, 1992; Miller, Templin, & Czerniak, 2011) and 
technology (Wei, Hung, Lee, & Chen, 201 1) to acquire new skills and knowledge. 

Recent teacher accreditation bodies, such as Pearson and edTPA and the Danielson Framework, with a focus on 
literacy, the hard sciences and assessment procedures, verify the need to produce elementary education teachers who 
are competent in the Science, Technology, Engineering & Mathematics (STEM) discipline. To be literate in the 
Global Age, citizens must be competent in mathematics and technology, and proficient in visual, communication 
and calculation skills in addition to “reading and writing” (Cooper & Krieger; Goldberg, 2006). 

A plethora of research evinces that free source ware (free ware) and Web 2.0 tools are multiplying at a rapid and 
steadfast rate over the last decade, are highly motivational and available in multiple content areas and disciplines. 
Thus the purpose of this study is to share effective teacher education strategies that utilize constructivist teaching 
methods and technology to increase skill level and content knowledge in elementary level mathematics and science 
- through the implementation of source ware and free Web hosting and Web 2.0 tools with a focus on Wikis. Free 
Wiki Web sites have increasingly gained popularity due to their ease of use, diversified multimedia and 
mathematical tools, and ample space for posting. Schools are feeling the budgetary pinch along with the rest of the 
country, so the use of free technology tools not only makes sense but is critically necessary as forecasters are 
predicting more cuts to education. 

In addition to lacking a surplus of elementary education teachers who are competent in mathematics and science, 
there is shortage of teachers who are trained in the Science, Technology, Engineering & Mathematics (STEM) 
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discipline. This research is an effort to help teachers and teacher candidates to both review and hone their content 
and process skills in the hard sciences and become equipped to use emerging tools in technology 

Research Hypothesis: 

H R1 : Teachers and teacher candidates who are exposed to motivational source ware Web sites (including free Web 
2.0 hosting sites) will implement these sites in their classrooms and increase their perceived skill/ability level in 
using new and emerging technologies. 


Methods 

Participants 

This study surveyed 76 American college students from two graduate level advanced capstone classes (one 
fully online and one blended) and two mixed undergraduate and graduate level student teaching classes. 
Approximately sixty-two percent (n=47) of the participants are certified teachers and thirty-eight percent (n=29) are 
teacher candidates. Majority of the students are graduate level (87%, n=66), female (83%, n=63), between the ages 
of 20-29 years (67%, n=44), and rate their perception of their skill level in technology as average (53%, n=24), to 
above average (36%, n=16), and expert (8%, n=4). 

Table 1. College Student Sample 


Students 

Male 

Female 

Undergraduate 

Graduate 

Certified 

Teachers 

Age Range 

20-29 years old 

7 

45 

8 

44 

32 

30-39 years old 

4 

10 

1 

13 

9 

40-49 years old 

0 

8 

1 

7 

7 

50-59 years old 

1 

0 

0 

1 

1 

60+ years old 

1 

0 

0 

1 

0 

Totals 

13 

63 

10 

66 

47 


n=76 


Instillments 

The self-administered questionnaire used for the analysis was the International Source Ware & Creativity 
Survey (O’Connor-Petmso, 2012). This survey instrument was pilot tested and has a Cronback alpha reliability 
coefficient of .785. The survey consists of 40 questions and is divided into three parts: Part 1) Background 
Information, Part II) Agree/Disagree set to a Likert Scale of 1) Strongly Disagree, 2) Disagree, 3) Non-Applicable, 
4) Agree, and 5) Strongly Agree, and Part III) Qualitative Responses. Present data were collected over a six-month 
period. 

Procedure 

The self-administered questionnaire is online and students were asked to take it either in class or at home. 

Analysis 

The research data were analyzed using IBM's PASW (Predictive Analytics Software), v. 22. Descriptive 
statistics and t-tests were run to ascertain frequencies and mean differences between teachers and teacher candidates. 
Results are reported including the participants’ recommended Web sites for elementary mathematics, science and 
Web 2.0 source ware. 
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Results 

Frequencies 

Similarities of Mathematics Source Ware Predilection by Teachers and Teacher Candidates (see Table 2) 

Approximately three-fourths or more of both the teachers and teacher candidates located math source ware and 
believe it helped them review basic facts and more than one-half of the participants believe it increased their skill 
level/abilities in math. Upon exposure to mathematics source ware, both teachers and teacher candidates plan on 
using this medium in their classroom - a 50% increase from their current use in the classroom. 

Table 2. Major Similarities of Mathematics Source Ware Predilection by Teachers and Teacher Candidates 


Variable 

Teachers 

Teacher Candidates 

V21: Found many free sites for teaching 
mathematics 

83% 

86% 

V22: Math source ware helped me 
REVIEW basic facts 

77% 

83% 

V23: Math source ware increased my 
skill level/abilities in math 

62% 

62% 

V24: 1 currently use math source ware in 
my classroom 

30% 

*21% 

V25: 1 plan on using math source ware 
in my classroom 

87% 

76% 


Note:* Teacher candidates are not necessarily implementing instructional methodology in the classroom and are 
limited by the technological infrastructure. 


Similarities of Science Source Ware Predilection by Teachers and Teacher Candidates (see Table 3) 

Approximately three-fourths or more of both the teachers and teacher candidates located science source ware and 
believe it helped them review basic facts, and approximately two-thirds believe it increased their skill level/abilities 
in science. Upon exposure to science source ware, both teachers and teacher candidates plan on using this medium 
in their classroom - a 50% increase by teachers and a 70% increase by teacher candidates from their current use in 
the classroom. 

Table 3. Major Similarities of Science Source Ware Predilection by Teachers and Teacher Candidates 


Variable 

Teachers 

Teacher Candidates 

V26: Found many free sites for teaching 
science 

74% 

72% 

V27: Science source ware helped me 
REVIEW basic facts 

72% 

76% 

V28: Science source ware increased my 
skill level/abilities in math 

66% 

65% 

V29: 1 currently use science source ware 
in my classroom 

26% 

*10% 

V25: 1 plan on using science source 
ware in my classroom 

79% 

83% 


Note:* Teacher candidates are not necessarily implementing instructional methodology in the classroom and are 
limited by the technological infrastructure. 


Similarities of Web 2.0 Source Ware Predilection by Teachers and Teacher Candidates (see Table 4) 

More than one-half of both the teachers and teacher candidates found wikis and free Web hosting sites such as 
Filamentality to be highly motivating and plan on using them in the classroom. Between 80-90% of teachers and 
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teacher candidates will continue to search for source ware in the hard sciences and web hosting sites and believe 
knowledge of these new and emerging tools is essential for the globally competent citizen. 

Table 4. Major Similarities of Web 2.0 Source Ware Predilection by Teachers & Teacher Candidates 


Variable 

Teachers 

Teacher Candidates 

V31: Wikis are highly motivating 

62%% 

68%% 

V32: 1 have created my own wiki 

51% 

*31% 

V34: 1 will use wikis in my classroom 

53%% 

55% 

V35: Free Web hosting sites like 
Filamentality are highly motivating 

64% 

59% 

V36: 1 will continue to search for math 
and science source ware and free Web 
hosting sites 

89% 

86% 

V37: In order for our citizens to be 
globally competitive, teachers must 
teach their students about Web 2.0 tools 
and source ware. 

89% 

83% 


Note:* Teacher candidates are not necessarily implementing instructional methodology in the classroom and are 
limited by the technological infrastructure. 


t-tests 

There were no significant mean differences between the grouping variable teacher and teacher candidates. 

Favorite Mathematics, Science, and Web Hosting Sites by Teachers and Teacher Candidates (see Table 5) 

The participants also recommended numerable favorite math and science source ware sites for the elementary 
grades. Table 5 lists the top nine free sites. Although the application Brainpop ( http://www.brainpop.com was listed 
as a top site for elementary science by the participants, it is not listed in the table because it is now proprietary 
software. 

Table 5. Favorite Mathematics, Science, and Web Hosting Sites by Teachers and Teacher Candidates 


Mathematics Source Ware 

Cool Math 

http://www.coolmath.com/ 


IXL 

http://www.ixl.com 


PBS Kids 

http://pbskids.org/aames/math.html 
and http://pbskids.org/cvberchase 


Math Is Fun 

http://www.mathsisfun.com/ 

Science Source Ware 

Science for Kids 

http://www.sciencekids.co.nz/ 


Bill Nye, The Science Guy 

http://www.billnve.com/ 

Web Hosting Source Ware 

Wikispaces 

http://www.wikispaces.com/ 


Filamentality 

http://www.kn.att.com/wired/fil/ 


Google Drive 

http ://www .aooale. com/dri ve/app s . html 


Discussion & Implications 

Globalization catapulted government, industry, and educational leaders into high stakes Information and 
Communication Technologies (ICT) planning in order to create “work ready” citizens for the global platform. 
However, due to an anemic economic recovery and continued educational budget cutbacks (Ginn, 2013), 
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infrastructures and applications will remain steadfast. One possible panacea is in the use of free source ware to keep 
our teachers and teacher candidates both motivated by new and emerging tools in technology and skilled in critical 
content areas. 

The participants of the survey overwhelmingly acknowledge the value of mathematics and science source ware 
and web hosting tools in that it is both motivating and offers them the tools to both review and hone their content 
and process skills in these disciplines - which is clearly needed and reverberated in the tenets of the numerable 
accreditation bodies which have usurped a large portion of the instructional agenda and assessment system within 
many Schools of Education in the United States. 

The results also show that 80-90% of both teachers and teacher candidates believe they will use free source ware 
in their current and future classroom. Thus the authors will proceed with a follow-up survey to assess the impact of 
these tools by both the teachers in terms of their instructional effectiveness and by the students in terms of increased 
achievement in mathematics, science, and technology. 
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Abstract 

The purpose of this study was to understand K-12 teacher perceptions on web-based tools for collaborative learning 
and how the perceptions affect the selection and utilization of the tools in teaching. An online survey was created to 
collect data and snowball sampling technique was used to recruit participants through Facebook. The results show 
that there is not strong correlation between participant perceptions and their utilization of the tools in teaching. 

Key words: K-12 teacher, web-based tools, perceptions, utilization 

Introduction 

Studies showed that attitude and mindset influence teachers utilizing Web 2.0 applications in instruction (Capo 
& Orellana, 2011, Luehmann & Frink, 2009). Teacher preconceptions can cause them to overlook the potential of 
web tools to engage students in more in-depth learning (Luehmann & Frink, 2009). The previous studies have mixed 
results on whether and how content areas and teaching level influence teachers’ intention and utilization of web- 
based tools in general (Sadaf et ah, 2012, Yuen, Yaoyuneyong, &Yuen, 2011, Parnell & Bartlett, 2012). The intent 
of this research was to investigate K-12 teachers’ perceptions and use of web-based tools with a narrower focus on 
collaborative learning. 

This study aimed to answer the following questions: 

1. What are K-12 teacher perceptions on web based tools for collaborative learning? 

2. Flow do teacher perceptions affect the selection and utilization of web based tools for collaborative learning 
in K-12 education? 


Literature Review 

Web-based tools 

Web 2.0, as a concept, as well as a set of tools DiNucci (1999), has been widely used and misused after a Web 
2.0 conference held by MediaLive International and O’Reilly Media, Inc. in 2004 (O’Reilly, 2005). Due to the 
difficulty in defining the concept “Web 2.0”, some of the authors clarify the term by identifying its features/ 
competency and a variety of applications that could be considered the set of tools that comprise Web 2.0. Some of 
the core features identified are interactive platform ((O’Reilly, 2005); improving information retrieval, relevance, 
and awareness (Barsky and Cho, 2007); harnessing collective intelligence (O’Reillyand Batelle, 2009); user sharing, 
contributions, and participation (Bonk, 2011). The tools that have been considered as Web 2.0 tools are Wikis, 
Blogs, Google, and podcasting (O’Reilly, 2005; Barsky and Cho, 2007; Norton and Flathaway, 2008). This study 
avoided using the concept “Web 2.0” and gives preference to the more cumbersome, but less ambiguous, term 
“web-based tools for collaborative learning.” While this term is overly broad, the focus will be on five categories of 
tools: blogs, social networks, wikis, podcasting, and video sharing, with specific applications identified in each 
category. These tools were chosen due to their accessibility, free-to-use options, potentials for cooperative learning 
and prior existence in literature. Throughout this study, researchers specify web-based tools by name and/or the 
categories in use. 

Blogs. Blogs (short for Weblogs) are online journals with one primary author to facilitate asynchronous 
communication on a particular topic. Blogs are part of many course management systems (Blackboard, Desire to 
Learn etc) and have been “viewed as the most useful Web 2.0 application in terms of improving student learning” 
(Ajjan and Flartshorne, 2008). In recent years, advances in mobile technology reduce the need for specialized 
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equipment and make it easier to add words, pictures, or video to blogs from handheld devices (Parnell & Bartlett, 
2012). Research show that blogs play positive role in improving learners’ acquisition of knowledge (Rosen & 
Nelson, 2008), enhance students' learning experiences and deepen levels of learners' engagement and collaboration 
(Maheridou, Antoniou, Kourtessis, and Avgerinos , 2011), facilitate rich interactions to create a dynamic community 
of learners with different areas of expertise (Chong, 2008), connect with peers, establish a community of practice, 
and support each other’s work thereby overcoming professional and geographical isolation (Ray & Hocutt, 2006). 

Wikis. Wiki is generally defined as a web page or site to which users and viewers alike can contribute 
and/or modify using either simple text markup language or a rich text editor (Peterson, 2009). Wikis are 
characterized by their collaborative nature, ease of use, convenience, open source access, uniqueness, and versatility 
(Shih et al., 2008). Teachers can use wikis to gain efficiency in distribution of information (Reich et ah, 2012). The 
most common use of wikis in a classroom setting is collaborative learning. This is made possible by features that 
allow users to create and share knowledge resulting in a rich, cooperatively created end-product (Biasutti and EL- 
Deghaidy, 2012). Moreover, the collaborative nature of wikis allows learners to develop a community and a social 
environment, wherein they create and edit content with and for each other. Pifarre and Staarman (2011) referred to 
this as the ‘thinking together’ model. When used with assignments that foster collaborative and group interaction, 
wikis can be rewarding learning tools that construct knowledge (Hazari, North, & Moreland, 2009). Wikis are also 
used to enhance collaborative writing and reflection for which students work together to achieve a common goal or 
objective as designated in their assignment (Wichadee, 2010). 

Social Network Sites (SNS). Social networking is web-based services that allow individuals to (1) 
construct a public or semi-public profile within a bounded system, (2) articulate a list of other users with whom they 
share a connection, and (3) view and traverse their list of connections and those made by others within the system 
(Boyd & Ellison, 2007). A survey conducted in 2009 by Schmucki, Hood, and Meell showed that Facebook and 
MySpace were two of the most popular social networking sites joined by and familiar to K-12 educators. Twitter 
found to be one of the top content-sharing tools for personal, professional, and classroom use (Schmucki, Hood, and 
Meell, 2009). 

Podcasts. “Podcasting is a means of publishing audio and video content on the web as a series of episodes 
with a common theme” (Deal, 2007, p. 2). It can be used to archive lectures, supplement course materials, and 
student created projects (Deal, 2007) or as alternative assessments (Pannell and Hutchison, 2010). 

Podcasts can be accessed and utilized anytime, anyplace, and have marked advantages in distance learning (Cook, 
2008). Students may utilize podcasts to fill in their notes with anything missed during classes, consequently 
resulting in additional review and more thorough notes (McDonald, 2008). McDonald also pointed out the 
usefulness of using podcast lectures to conduct self-assessment on the quality of the lecture and/or to adjust anything 
in the future. According to Borja (2005), students are motivated by the creativity and the broader audience that can 
be reached with podcasting. Pannell and Hutchison (2010) stated the benefits of podcasting on student motivation 
as students in a last class of the day were observed being actively engaged while listening to podcasts discussing a 
math assignment. Fryer (2008) stated “The fact that podcasts can be educational and instrumentally valuable in 
teaching students a variety of important twenty-first literacy skills, while also being fun, is more than icing on the 
cake” (p. 1). 

Video-sharing. Video-sharing and video-sharing sites allow users to view, upload, share, tag, and 
comment on posted videos. Chunneng, Tianjun, and Hsinchun (2010) indicated that videos on video-sharing sites 
can be user-generated, are short in length, and provide diverse content. “In the current decade, the technologies of 
interest are still collaborative ones, but also those that foster student generation and visualization of knowledge,” 
(Bonk, 2008, p. 6). YouTube, as the world’s most popular video sharing site, was chosen as the representative of 
this category in this study. Video sharing sites have many practical applications for K-12 use. “YouTube provides a 
platform for students to be content creators, not just content viewers,” (Adam & Mowers, 2007, p. 22). Bonk (2008) 
suggested the use of videos from video sharing sites as starters and enders of lessons and to motivate and engage 
students in their learning. He cited the ease of access and flexibility as further reasons to integrate videos from 
video-sharing sites. “The class comes to life and ideas begin to resonate with students,” (Bonk, 2008, p. 6). Bonk 
(2008) also stated that “online videos such as YouTube videos can augment or illuminate the weekly assigned 
readings,” (p. 5). Bonk (2008) indicated that videos “rouse students’ minds to life by showing them new insights, 
perspectives, and situations to learn from,” (p. 6). In addition to Bonk, Adam and Mowers (2007) stated that 
“YouTube can be a valuable resource and is just one more example of the potential of Web 2.0 tools, which, in the 
hands of enlightened educators, can inspire students and support their digital learning style,” (p. 22). 
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Methodology 


Participants. The target population for this study was K-12 teachers that utilize Facebook. Participants were 
invited through Facebook wall postings from their friends, which informed them of the purposes, methods and 
provided a link to the survey. The study was conducted in the spring of 2013. Sixty eight teachers completed the 
survey. 12 were male (17.6%) and 56 were female (82.4%). Ages ranged from under 25 to the 55 - 64 range. There 
were no responses from age 65 and over. Years of experience ranged from under 5 years to 26 years or more. Table 
1 below showed the characteristics of the participants. 


Age and years of experience of participants 


Age 


% 


Years of experience 

% 

Under 25 

5 

7.4% 


5 years or fewer 

16 

23.5% 

25-35 

20 

29.4% 


6-10 years 

18 

26.5% 

35-44 

20 

29.4% 


1 1-15 years 

13 

19.1% 

45-54 

9 

13.2% 


16-20 years 

6 

8.8% 

55-64 

14 

20.6% 


21-25 years 

6 

8.8% 

total 

68 



26 years or more 

9 

13.2% 





total 

68 



Table 1. Age and years of experience 


Data collection and analysis. An online survey was created by the researchers using Google Survey tool to gather 
and store data. The survey consisted of two parts: demographic information and teachers’ perceptions of the use of 
blogs, wikis, social networks, podcasts, and video-sharing tools. The questions about perceptions were grouped 
according to each web-based tool and each grouping had seven questions specifically geared to capture individuals’ 
responses to that specific tool. The questionnaire was closed-ended, Likert scale format (Strongly Disagree, 
Disagree, Neutral, Agree and Strongly Agree). The snowball sampling technique was used to recruit participants 
through the social networking website, Facebook. A Facebook group named “Please Flelp Me Find K-12 Teachers” 
was created by the researchers. Detailed description and the link of the survey were published on the group posting 
wall. The Facebook group was popularized among the researchers’ current Facebook friends, who were asked to 
share the group link and information to their friends and other potential participants through Facebook wall postings. 
Data collection was concluded after 12 weeks. 


Results and Discussion 

The purpose of this study was to examine K-12 teacher perceptions of web based tools for collaborative learning and 
how these perceptions affect the utilization of the tools both within and outside of the classroom environment. 
Conclusions are based on the self-reported frequency of tool usage and the Likert scale correlation of survey 
responses. 
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Tool usage 


Tables 2 and 3 show how often the participants utilizing the various tools for classroom activities (Table 2) and 
outside of the classroom (Table 3). 


In class usage 


Never 

Very 

Rarely 

Sometimes 

Frequently 

Very 

Frequently 

Blogs 

42.6% 

23.5% 

23.5% 

4.4% 

5.9% 

Wiki 

42.6% 

19.1% 

22.1% 

8.8% 

7.4% 

Social 

networks 

44.1% 

20.6% 

14.7% 

10.3% 

10.3% 

Podcasts 

42.6% 

19.1% 

26.5% 

10.3% 

1.5% 

Video 

sharing 

14.7% 

10.3% 

35.3% 

23.5% 

16.2% 


Table 2: Percentage of usage in classroom activities 


Outside of class usage 


Never 

Very 

Rarely 

Sometimes 

Frequently 

Very 

Frequently 

Blogs 

27.9% 

19.1% 

26.5% 

16.2% 

10.3% 

Wiki 

38.2% 

19.1% 

23.5% 

10.3% 

8.8% 

Social 

networks 

17.6% 

5.9% 

2.9% 

26.5% 

47.1% 

Podcasts 

42.6% 

17.6% 

20.6% 

14.7% 

4.4% 

Video 

sharing 

14.7% 

14.7% 

23.5% 

26.5% 

20.6% 


Table 3 : Percentage of usage outside of the classroom 


According to the results of the survey the most commonly used online collaboration tool outside of the classroom is 
Social Networking. This is not surprising since the survey was shared and accessed via a Facebook page. What is 
surprising is that 1 7.6% of participants responded that they never use Social Networks. This would indicate a flaw in 
the data collection methodology since participants were either able to access by other means or the definition of 
Social Network or Usage frequency was not clearly defined in the survey. 

Of greater interest was the in-class usage of Video Sharing Tools. This was the only tool which the majority of 
participants indicated a Sometimes or more often usage. All other tools were used Rarely or Vary Rarely by the 
majority. In all cases more than 40% indicated that they never use the tool in class. 

For graphical comparison, responses on the Likert scale were converted to numerical values. This ranged from 1 for 
Never or Strongly Disagree to 5 for Very Frequently or Strongly Agree. Figure 1 shows a graphical representation 
comparing the usage frequency average of the various tool by participants. 
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Average rating of how freqently these tools are used 

(1= never; 5 = Very frequently) 


5.000 


4.000 


3.000 


2.000 


1.000 



I utilize for classroom related activities. I utilize for non-school activities. 


■ Blogs 

■ Wiki 

■ Social networks 

■ Podcast 

E Video sharing tools 


Figure 1: Graphical representation of usage 

Tool perceptions 

The survey asked participants to rate their agreement with the following statements for each of the five tools (Blogs, 
Wikis, Social Networks, Video sharing and Podcasts). 

• Does the tool help the learners develop language and communication skills 

• Does the tool facilitate feedback between learners and teachers 

• Does the tool develop skills needed in today’s modern technology world 

• Does the tool promote learner interaction and build a learning community 

• Does the tool provide collaborative learning opportunities 

Figure 2 shows a comparison of teacher perception of the utility of each of the tools. Video sharing tools and Blogs 
were consistently ranked highly in each category while Social Network tools were nearly always the lowest ranked 
in each category. 


Average rating of percieved tool utility 

(1= stongly dissagree; 5= strongly agree) 



Help learners develop Facilitate communication Develop skills needed in Promote learner Provide collaborative 

communication and and feedback between today's modern, interaction and build a learning opportunities, 

language skills. learners and teachers. technological world. learning community. 


■ Blogs 

■ Wiki 

■ Social networks 

■ Podcast 

■ Video sharing tools 


Figure 2: Perceived tool utility 
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Correlations 


Table 4 shows the correlations of responses to all questions. The factor that most highly correlates with use of a tool 
in class is if the participant uses the tool outside of class. 


Overall Correlation of responses 




Help 

Facilitate 

Develop 






learners 

communicati 

skills 

Promote 

Provide 




develop 

on and 

needed in 

learner 

collaborat 




communica 

feedback 

today 's 

interaction 

ive 



Use 

tion and 

between 

modern, 

and build a 

learning 


Use in 

out of 

language 

learners and 

technologic 

learning 

opportunit 


class 

class 

skills. 

teachers. 

al world. 

community. 

ies. 

Use in class 

1 







Use out of class 

0.5490 

1 






Help learners 
develop 

communication and 
language skills. 

0.4732 

0.2235 

1 





Facilitate 

communication and 
feedback between 
learners and 
teachers. 

0.4164 

0.2196 

0.6308 

1 




Develop skills 
needed in today’s 
modern, 

technological world. 

0.4483 

0.2801 

0.6716 

0.6174 

1 



Promote learner 
interaction and build 
a learning 
community. 

0.3911 

0.2285 

0.6466 

0.7195 

0.6577 

1 


Provide 

collaborative 

learning 

opportunities. 

0.3979 

0.2687 

0.6843 

0.6667 

0.6525 

0.7814 

1 

Table 4: Overall correlations 
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Table 5 shows the correlations of classroom usage and the perception of utility of each tool. In general, the trend of 
highest correlation of in-class usage with non-school usage continues. The exception is in Wiki usage. For this tool, 
the highest correlation of in-class usage is with the instructor’s perception of how useful it is in developing 
communication and language skills as well as its ability to facilitate communication between learners and the 
instructor. These findings are similar to Sadaf, et al. (2012), who found that teacher’s likelihood of using web based 
tools in their classrooms highly depended on whether they thought it had the potential of positively impacting and 
improving students learning and engagement. 


Utilization for classroom related activities 


Blogs 

Wiki 

Social 

Networks 

Podcasts 

Video 

Sharing 

1 utilize for non-school activities. 

0.6494 

0.4557 

0.4696 

0.6426 

0.6882 

Help learners develop 
communication and language skills. 

0.3509 

0.6608 

0.4892 

0.4704 

0.3960 

Facilitate communication and 
feedback between learner and 
instructor 

0.3033 

0.6103 

0.5315 

0.3504 

0.3062 

Develop skills needed in today’s 
modern, technological world. 

0.2627 

0.5918 

0.4356 

0.4442 

0.4610 

Promote learner interaction and build 
a learning community. 

0.2329 

0.5114 

0.5312 

0.3685 

0.3257 

Provide collaborative learning 
opportunities. 

0.2450 

0.4806 

0.5016 

0.4417 

0.3515 

Table 5: Correlation by specific tool 


Conclusion 

The result of this study shows that the factor that most highly correlates with K-12 instructor usage of tools 
within the classroom is the instructor’s usage of the tool outside of the classroom. Therefore, the familiarity with 
tool outside the classroom and prior experience with such tools are determining factors on whether or not an 
instructor will use the tool in his/her classroom. These findings corresponds to Yuen, Yaoyuneyong, and Yuen 
(2011), who found that teacher’s additional experience using a given tool increases his/her perception and likelihood 
of using the tool in the classroom. It is recommended therefore, that departmental administration, policy makers and 
educators strive to provide more professional development and an environment that allows teachers to integrate 
these resources in their classrooms. Similar studies share this suggestion and show that teachers experience and 
proper guidance in the use of new technologies gives them confidence to help their students explore these tools 
(Ajjan and Hartshorne, 2008; Ertmera, et al. 2012; Yuen et al. 201 1). Past surveys lumped all tools together and 
referred to them as “Web 2.0”. This may overly simplify the results given the variation in the perceived utility of 
the tools seen in this study. 


Limitations 

There was a very low response rate to this survey. There were only 68 useful respondents despite extending the 
collection period to twelve weeks from the planned six. Others have had greater success utilizing Facebook to 
contact potential participants. The concept was to facilitate an expanding network of contacts. Based on the limited 
number this was unsuccessful. In addition, this method of contacting potential participants creates an inherent bias in 
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the representation of the opinions of users of Social Network sites. A preferred method would be to utilize each tool 
for the initial contact or an alternative method that with a more equal tool utilization distribution. 

For all tools except Video Sharing, most respondents indicated that they rarely or never use the tool. While the goal 
of this project was to measure instructor’s perceptions of collaborative tools, this lack of familiarity is a confounding 
issue. 


Areas for further study 

Given the low correlations between teacher perception of the usefulness of the tools and the usage in the classroom 
none of the factors identified in survey are strong driving factors in adoption. There were additional questions in the 
study this one was based on (Yuen, Yaoyuneyong, & Yuen 2011) that were omitted in an effort to create a more 
targeted survey. It is possible that this effort to create specificity eliminated relevant factors. Areas of consideration 
are cost of adoption, department or administration directives regarding the tool and technical challenges associated 
with incorporating the tool in the curriculum. 
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Introduction 

The field of education has exhibited two significant trends in the past decade. First, there has been 
increasing attention to the close relationship between emotion and cognition in the processes involved in human 
learning. Second, online educational technologies such as course management websites have become more 
widespread. Numerous studies have been published in both areas in the past decade. However, less attention has 
been given to merging these areas of studies. The purpose of our study is to link these two areas of focus by 
answering the question of how human emotions affect cognition in a challenging online environment. 

More specifically, we are interested in how emotions affect cognition when individuals are completing the 
online application for approval to conduct research with humans, what on our campus is called the IRB process. In 
this research, we tracked the influence of emotion on cognition and decision-making, bringing in aspects of 
educational psychology constructs such as motivation, knowledge construction, and information retrieval. Since 
online tools have become more prevalent in education, there has been increasing attention to the need to understand 
and facilitate cognitive activities in online environments. As a result, online web design rubrics have been created 
guided by cognitive principles to produce effective online educational tools. However, there has been less attention 
devoted to understanding best design issues of other applications such as websites that individuals must navigate to 
fulfill their goals. 
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Many online applications involve more than simply filling in blanks, reading text, or watching videos. Once 
a web page opens, it presents to the user several possibilities for his/her attention and action, as for example, the 
navigation menu bar, columns with large amounts of text, many buttons to be clicked, and hyper-links that direct the 
user to other pages (Coiro & Dobler, 2007). The online task can be interpreted as an occasion of continuous 
cognitive activity that requires a great deal of attentional resources (Mayer, Moreno, Boire, & Vagge, 1999), to 
which we added a concern for the emotional demands of the task. In other words, emotions, attitudes, and 
environmental factors such as frustration, worry, and exasperation may affect the decision-making and cognitive 
learning during the online activities. This kind of web application demands sophisticated mental strategies for 
navigating, focusing attention, evaluating, and synthesizing text that requires literacy skills, strategies that might 
place undue cognitive load on users if the online content is not effectively designed. This paper aims to address the 
importance of the role of psychological phenomena in online environments that emphasize cognitive decision- 
making and emotions, how they affect each other, and their implications for online activity. 

Literature Review 

Cognitive Overload and Emotions in Web Environments 

Cognitive overload is an important aspect of many cognitive activities, particularly one that involves the 
display of many bits of information and the need to decide what action needs to be taken, or when a user accesses an 
unfamiliar website with which he or she needs to interact actively (Mayer et ah, 1999). Such interaction can be 
highly affected by the user’s emotional arousal. An early article by Vigil (1983) emphasized the limits of human 
memory in continuous processing and searching for relevant information. When a large amount of information is 
provided to online users that they must interpret cognitively, it creates cognitive overload. This engenders a state of 
cognitive overload as they attempt to understand the large amount of information they see and to decide what they 
must do. In the circumstance of explaining the relationship between online content and cognitive overload, a 
cognitive theory of multimedia learning can be applied to explain cognitive overload. Mousavi, Low, and Sweller 
(1995) argued that a split-attention effect occurs when the text is given with the animation so that learners must split 
their visual attention to the text and the animation. In such situations, learning can be interrupted since the 
integration process exceeds the limit of working memory capacity. 

Vigil also addressed how continuous effort to interpret relevant information from a large amount of content 
results in anxiety and confusion for users. Eysenck, Derakshan, Santos, and Calvo (2007) argued that anxiety can 
interrupt functioning of goal-directed attentional processes. Moreover, anxiety diminishes attentional control. We 
were interested in how ill-designed multimedia contents might cause cognitive overload and negative emotions such 
as anxiety and frustration. As a result, smooth progression in using or learning from multimedia contents might be 
impeded. 


Success in any online activity seems to depend on how well a website designer has anticipated and found a 
way to translate process features into a symbol and structured system that users can understand, and then delivered 
this information through technology (Clark, 2009). If the designer does a good job of this cognitive translation, the 
user can learn from and make use of the application. 

Emotions and Cognition in Goal Attainment 

The increasing interest in students' emotional experiences has led to a more sophisticated understanding of 
how emotions and motivation interact and influence task engagement (see, for example, Pekmn, 2006). However, 
little attention has been paid to the role of such psychological factors as emotion and mood in how students perform 
online activities (but see Marchand & Gutierrez, 2012; Tempelaar, Niculescu, Rienties, Gijselaers, & Gisbers, 2012). 
Several studies support the claim that emotions like joy, pride, hope, sadness, anger, fear, and boredom influence 
thought, action, and decision-making (e.g.. Do & Schallert, 2004; Izard, 2007; Pekmn & Stephens, 2009; Shiv, 
Loewenstein, Bechara, Damasio, & Damasio, 2005). In addition, Meinhardt and Pekrun (2003) demonstrated that 
emotions affected the processing of cognitive information by increasing information-processing load and draining 
attentional resources from the task. 

Pekmn, Goetz, Titz, and Perry (2002) proposed a control-value theory of achievement emotions, describing 
how emotions influence task deployment based on appraisals an individual makes of whether he/she has control 
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over task success and whether the task is valuable. When the person perceives the task as controllable and important, 
positive emotions such as hope and pride are likely to be experienced. Conversely, a sense of lack of control or a 
negative subjective valuing of the task is expected to produce negative emotions such as anxiety, hopelessness, or 
shame. For example, anxiety may be aroused if the person begins to expect failure on an important task but 
perceives it as not sufficiently controllable. In Boekaerts’s (2002) view, such intense negative emotions are likely to 
encourage individuals to regulate their emotions for the purpose of restoring well-being by seeking social support. 

A second line of work comes from research on goals. Any interaction with an online interface depends on 
individuals’ goals (e.g., Kwon & Schallert, 2012). Simons, Dewitte, and Lens (2004) explained that goals can be 
multiple, complex, and defined as cognitive representations of future objects/outcomes that individuals are 
committed to approach or to avoid. Carver and Scheier (1998) described how emotions may signal whether or not 
sufficient progress toward a goal is being made. Pintrich (2003) identified two approaches researchers have taken to 
describe how goals influence students, the prolific work on achievement goals and the less extensive literature on 
content goals (Ford, 1992; Wentzel, 2000). The latter seemed particularly relevant to describe what influenced our 
participants as they strove to finish their task. 

Finally, the work on the role of prior knowledge was relevant to our project. It is well-established that 
individuals with more relevant experiences can accomplish a task more effectively than those with little or no 
experience (c.f., Alexander, Schallert, Hare, 1991; Waters & Waters, 2010). Because novices approach a task in a 
more surface, cursory manner than experts, they often show poor problem-solving and decision-making (de Jong & 
Ferguson-Hessler, 1986, 1991). In our project, we examined the emotional experiences of more and less experienced 
users of a web application as they interacted with the program to fulfill an important goal. 

Research Questions 

With the overarching goal of understanding better how emotions influence cognition in the authentic 
context of interacting with an online application, we addressed the following questions: (1) What features of a web 
application cause emotional arousal that affect cognitive responses such as decision-making process and reading 
comprehension as they work to complete the application? (2) How do novice users of an application differ from 
experienced users in their emotional and cognitive responses to the online task? 

Method 

Participants 

Participants were six doctoral students (four women, two men; two international students with advanced 
control of English) enrolled in a social science program. The students were recruited as they were about to submit an 
application for approval from the university’s review board charged with protecting the ethical treatment of research 
participants. Four were planning to submit their first IRB application, the “novices,” and two were “experienced” 
(with three or more successful IRB applications). 

Website Context 

The IRB online web application is mandatory for anyone who wishes to conduct research with human 
participants. Finding the actual start of the application is itself not obvious from the home page of the IRB website. 
The application itself consists of six steps that researchers must complete regarding their study by responding to 
items of different types, including drop down menus, open-ended questions, and multiple-choice sections. In 
addition, there are a few hyper-links that provide additional information for some of the questions and sections, 
ostensibly meant to help a researcher make an appropriate determination in answering a particular question. 

The site is made difficult by a few buttons that a user should recognize as navigation buttons to move to 
different steps, with save buttons located at the bottom of each page (or step), and file upload buttons that are subtle 
and located at the top of the page. When a user makes a mistake or forgets to fill out a question, a red warning sign 
appears at the top of the screen until the problem area is fixed and saved. In order to provide users with much 
information, the font is very small and sometimes presented in a color that does not have good contrast with the 
background. Some additional instructions that are for optional questions appear in red so that they catch more 
attention than the actual questions that must be answered. 
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The last step of the application requires that the researcher upload a word file of the proposal that repeats 
all the same information that has been used to fill out the steps of the application. At this point, a message is sent to 
the researcher, the faculty sponsor of the research (if the researcher is a student), and to the department-assigned 
IRB staff who will then make some preliminary determination of whether the application has been filled out 
appropriately. 

Procedure for Data Gathering 

Data were collected in two sessions: a video/audio-recorded "work session" in which the participant 
worked on the IRB application, and a post-interview session that used the video-record for stimulated recall. For the 
first session, participants were instructed to narrate aloud what they were thinking and feeling during the activity. 
Camtasia software captured each participant's desktop and mouse pointer movements. Cameras simultaneously 
captured video and audio of the participant's facial expressions and speech. The interview sessions were audio- 
recorded and later transcribed. 

Data Analysis 

Analysis proceeded in four phases. First, we made a transcript of the first session for each participant, 
screen-capturing their faces whenever they spoke aloud, took action as they interacted with the website, or showed 
some sort of emotion. We refer to these freeze shots as frames. We tracked time spent in different episodes of the 
application. Second, we labeled the emotions each participant evinced, with two research team members working on 
each transcript to reach consensus on any discrepant decision. Third, to these transcripts of the first session, we 
connected interview transcript segments. We used participants’ words to determine what emotions they had 
experienced and how emotions played a role in their decision-making. Having analyzed for the first three 
participants all steps of the online IRB application, we noted that steps 1 and 4 were particularly critical in the 
process and caused participants much difficulty. We therefore limited our analysis to these two steps for the rest of 
the participants. Our final phase involved deriving themes to capture the phenomenon and returning to the data to 
look for confirming and disconfirming evidence. 


Results 

What the participants were attempting to do in filling out their IRB application can be described as a 
problem-solving or decision-making activity that required interpreting dense text delivered on a website with some 
infelicitous aspects (e.g., no warning that a step had not been saved). Although some links were provided to 
resources from other websites, these were often not any friendlier as a source of guidance (e.g., Federal guidelines 
and descriptions of the law that applied). In addition, the task required the user to make a series of decisions about 
how his/her research fit the descriptions provided. In the face of such a difficult, resource-demanding cognitive task, 
all participants showed some degree of negative emotions when they encountered a decision point but did not know 
how to proceed. We organize our presentation of results in two parts by addressing each of our research questions. 

Part 1: What features of a web application cause emotional arousal, thereby affecting cognitive responses? 

We addressed this research question by presenting the result of three steps in our analysis. 

(1) Categorizing problematic issues. As mentioned earlier, an examination of all data sources led us to 
identify three basic types of sources of problems or difficulties as participants worked with the IRB online website. 
One type, labeled psycholinguistic issues, referred to barriers identified by the participants or suggested by their 
actions that seemed to originate in the wording on the page, either because the words used were unfamiliar or 
because the participant could not determine to what the words referred. For example, several participants struggled 
with the words and their associated concepts when they had to decide whether their study fell into the category of an 
exempt , an expedited, or a full board study. 

A second type, which we labeled interface design issues, referred to examples where the participants 
seemed to be stymied either by not knowing how to save the current step in the application, by not understanding 
what an error message was asking them to do, or by other aspects of design such as font size, font color, and text- 
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heavy sections. For example, several of our participants did not recognize how to save a step, and thus, when they 
went on to the next step, they lost all the work they had done. 

A third type of problems participants encountered we saw as an “other” category. We labeled this category 
psychological issues, and used it whenever participants expressed that they disagreed with how they were being 
asked to provide information or when the application asked information in such a way that they experienced a 
mismatch with their expectations or prior knowledge. Examples included when they were asked to list information 
that already appeared in the proposal document they knew they would also upload or when they were asked if their 
study made use of human tissue, something they felt should have already been precluded by information they had 
provided at earlier steps in the application. 

(2) Identifying the emotional responses that occurred in response to different online features. The 

results of data analysis indicated that participants experienced and expressed emotional responses frequently as they 
went through the online application. As shown in Table 1, we categorized the emotions individuals expressed into 
two major categories, the annoyed/frustrated category and the confused/worried/puzzled category. These two large 
categories subsumed several different variants and particular emotion labels. 

Note that there were in the data occasional expressions of positive emotions. However, because our focus in 
this report was on how participants responded to difficulties and problems they encountered as they worked with a 
website, we found that no instance of a positive emotion was associated with problems, not even relief or pride upon 
having solved a problem. The closest to a positive emotion came from an experienced user who was neutral and 
resigned whenever he was not certain of what he should do, relying on his knowledge that he would receive 
feedback from the IRB staff letting him know the mistakes he had made. 

Having assigned each emotion expression to either of the two major categories, we next assigned a rating of 
mild or strong based on facial expression, concurrent think aloud words, and post-interview statements. We then 
tallied the number of emotion expressions concurrent with each kind of problem, psycholinguistic, interface design, 
or psychological issues (see Table 1). 

As shown in Table 1, participants experienced emotions at about the same frequency for the three kinds of 
problems they encountered. Also, even if novice users differed from experienced users in their underlying 
understanding of the overall web application process, both novice and experienced users expressed frustration and 
even anger, as well as anxiety and worry about whether they would finish the task. For all participants, when it came 
to being confused or puzzled or worried, the intensity of their emotion seemed more often mild than strong, at a ratio 
of about 2 to 1 . When it came to being annoyed or frustrated, they more often experienced the emotion to a strong 
degree rather than to a mild degree, again at a ratio of about 2 to 1 . 


Table 1. Fre q uency of Expressions of Mil d or Strong Emotions Associated with Dif f erent Problems 


Participants/Emotions 

Psycholinguistic Issue 

Interface Design 
Issue 

Other Psychological 
Issues 

Pseudonym 

Emotions 

Mild 

Strong 

Mild 

Strong 

Mild 

Strong 

Evan 

Frustrated 

2 

4 

2 

3 

1 

3 

Confused 

3 

3 

2 

2 

1 

1 

Kim 

Frustrated 

0 

1 

2 

4 

1 

4 

Confused 

7 

3 

5 

3 

2 

2 

Eunyoung 

Frustrated 

2 

1 

1 

6 

2 

0 

Confused 

4 

1 

0 

0 

5 

2 

Steven 

Frustrated 

0 

2 

3 

4 

3 

4 

Confused 

2 

2 

4 

2 

2 

2 

Anna 

Frustrated 

0 

1 

1 

6 

0 

0 
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Confused 

4 

3 

2 

0 

4 

1 

Jayoung 

Frustrated 

1 

2 

3 

6 

4 

4 

Confused 

5 

1 

2 

4 

5 

5 

Total 


30 

24 

27 

40 

30 

28 


(3) Emotions were intertwined with cognitive processes, especially when a decision seemed 
particularly consequential to goal attainment. As mentioned above, the emotion of being confused or worried 
was more often expressed as a mild than as a strong emotion whereas the reverse was true for frustration or 
annoyance. Interestingly, whether mild or strong, participants seemed to spend as much time in the cognitive activity 
associated with resolving the problem, whether it was to comprehend what they should do, remember what they had 
done earlier in the application, or decide which alternative among several applied to their study. Psycholinguistic 
and other psychological issues seemed to take a long time to resolve. By contrast, interface design issues seemed to 
lead more often to strong emotions that more quickly abated (true of 5 of our 6 participants). 


For example, Anna, a more experienced user, showed mild confusion, strong surprise, strong worry, and 
mild frustration as she filled out the application. In the first photo, when she was trying to decide whether her study 
was an expedited or exempt project, she expressed puzzlement as to what it would mean to pick either alternative, 
quickly choosing that her study must be an expedited study. In the second panel, her surprise came when an error 
message identified that her IRB training was not up to date, even though she had completed the training several days 
before filling out the application. She spent several minutes and expressed quite a bit of frustration that she was not 
approved. The third panel came when she overlooked updating a step and lost all the changes she had made to the 
application. Finally, as she saw the question about whether women of childbearing age were to be included in the 
study, she expressed frustration and confusion that a study involving an online survey about students’ views of 
foreign language teaching would somehow have anything to do with a pregnancy test. 



type of 

study she was describing, and about whether she was the principal investigator of the study, and then frustration and 
even anger about being asked whether the study included the use of radiation, about one question that was marked as 
obligatory and yet would only be answered if one had said yes to the previous question, and about the section where 
she needed to decide which of three sections related to the consent form she should answer. 



Evan, also a novice user, was frustrated by the wording of questions such as, “if student project, please 
describe,” to which he responded “Describe what??!! Describe my hobbies and interests? Please describe my ideal 
date? What the f— are you asking?” Replete with strong words and many expressions of frustration at why an error 
message was not going away or what it meant to identify the study as a single site or multi-site study, he spent 
almost one hour on the application but never finished the actual task. We conjectured, and confirmed in the post- 
interview, that some of the difficulties he encountered in finding the button he needed to click to go on to the other 
step were caused by his frustration levels. 
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Part 2: How did novices differ from experienced users in their emotional and cognitive responses to the online 
task? 


For novices, emotions tilled the experience because many decisions about what they should do seemed 
fraught with uncertainty. For example, when Evan came to step 4, the section where he needed to choose among 
three sets of questions about the use of consent forms, he expressed annoyance and confusion at the fact that the font 
was in three colors, not ever realizing that he needed only to answer one of the sections. He read each section aloud 
and provided answers for all three alternatives even though he commented on the logical inconsistencies he saw. 
Vacillating between going off-task to chat with friends and cursing as he re-read the sections (“What the f— is the 
difference between privacy and confidentiality?”), he eventually spent 15 minutes on just this section alone. 

Although the two experienced users also displayed negative emotions, these seemed more mild than the 
intense negative emotions expressed by novice users. Also, the few expressions of neutral and positive emotions 
came from the experienced users rather than novices, suggesting that for them the task was a milder, more routine 
aspect of doing research. Unexpected changes in the web application could jar experienced users, causing them 
momentarily to act like novices; after some time had passed, they could more easily than novices re-engage the task 
with the smooth, calm demeanor they usually displayed. Jayoung, surprised by the fact that the website had changed, 
took some time to self-regulate to re-engage with the task, using her more extensive knowledge and her generally 
solid understanding of how the process would unfold to recover from her frustration. 

Discussion 

Our study demonstrated that emotions, when aroused by the very task of filling out a difficult application 
online, could affect cognitive process such as decision-making and comprehending information. These emotions 
were powerful and had ramifications on the process of interacting with the website such as adding time to the 
process and interfering with decisions that needed to be made. This was true even though every participant in our 
study was a frequent and extremely competent user of multiple websites, true millenials and quick learners in their 
fields. We were surprised by how easy it was to witness various emotions expressed at various levels of intensity 
from individuals who were all intent on finishing the task successfully. 

Previous psychological research on web interface design has approached the problem as a cognitive-based 
issue, identifying instructional or interface design problems that cause cognitive overload on the user and lead to 
difficulties in interacting with an online interface (see Figure 1). Our results suggested that emotions were a critical 
component resulting from three kinds of sources, psycholinguistic comprehension problems, interface design issues, 
and other psychological issues based in the user’s expectations and prior experiences. When encountering such 
issues, strong emotions could arise that themselves contributed to the cognitive overload in the situation, and that 
then influenced subsequent cognitive processes such as decision-making, comprehension, and memory (see Figure 
2). The contribution of our work to the previous conceptualization of difficulties encountered by users is to highlight 
the important role that emotions play in human experience. 
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Figure 1. Model of traditional cognitive study of difficulties arising from web interface 




Figure 2. Model from our results showing how emotions contribute to difficulties arising from web interface 
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Multimedia Learning: Effects of Background Music and Sound Effects on 

Space Science Concept Learning 


Vanthanh Phan 
Jongpil Cheon 


Abstract 

This study investigated the effects of background music and sound effects with audio narration on space 
science concept learning. Participants were randomly assigned to three groups: (a) commercial music background 
and sound effects, (b) Baroque music background without sound effects, and (c) no music background and sound 
effects. The study’s findings provided appropriate ways of including music background into multimedia 
instructional materials for space science concept learning. Students learned better when there was no background 
sound and music; however, they were more engaged in the instructional video that had commercial background 
music and sound effects. 


Introduction 

Music and sound effects have been widely used as background in video production, which is a great 
resource for teaching and learning. The rationale for this practice is based on arousal theory (Weiner, 1990), which 
states that interesting music and sound effects make multimedia message more enjoyable to the learners; therefore 
foster their levels of emotional arousal and learning. Learning will be improved when multimedia presentation 
include background music than in the silent condition (Gao, Chang, Ren, Aickelin, & Wang, 2010; Groot, 2006; 
Schon et ah, 2008; Thiessen & Saffran, 2009). Besides, background music was claimed to be a relaxation therapy 
for learners’ mind by its unique potential of rhythm to energize and aid information processing in the mind (Gatson, 
1968). Adding Baroque music as background was believed to help the body relax and the mind alert; therefore lead 
to super-learning (Ostrander, Schroeder, & Ostrander, 1979). 

However, Mayer (2009), who based his approach on knowledge construction and cognitive load theory, 
argued that adding irrelevant music and sound effects tends to have detrimental effects on learning outcomes. His 
experimental research supported his claims, which led him to coin the coherence principle for multimedia learning - 
learners learn better when interesting but irrelevant music background and sound effects are excluded from 
multimedia instructional materials. Many other researchers also found evidences supporting the coherence principle 
and claimed music background and sound effects as seductive details resulted in poorer retention and transfer 
performance (Garner, Gillingham, & White, 1989; Harp & Mayer, 1997, 1998; Mayer, et ah, 1996; Renninger, Hidi, 
& Krapp, 1992; Salame & Baddeley, 1989). 

Regarding the neutral cases, which neither support nor disapprove the use of background music and sound 
effects, Bmnken, Plass, and Lautner (2004) found the auditory cognitive requirements of background music alone 
did not differ from the auditory cognitive requirements of the materials without any auditory stimuli. Likewise, 
background music neither enhances nor decreases learning performance (Jancke & Sandmann, 2010). Meanwhile, 
other studies showed partial evidence for the hypothesis that music would facilitate retention and transfer 
performance (Miller & Schyb, 1989; Thaut & I’Etoile, 1993). 

These confounding research findings call for further research on whether and how background music 
should be embedded in multimedia learning. Conceivably, the key question remained is to find the appropriate types 
of music in multimedia learning (Mayer, 2009; Ostrander, Schroeder, & Ostrander, 1979). 

Method 


Participants 

In this study, there were two experiments designed to test the effects of background music and sounds on 
space concept learning. There were 68 participants in the first experiment (Male: 36, Female: 32; Freshman: 2, 
Sophomore: 6, Junior: 31, Senior: 29) and 74 participants in experiment 2 (Male: 38, Female: 36; Freshman: 2, 
Sophomore: 7, Junior: 33, Senior: 32). Participants were recruited from a southwestern university in the United 
States. 
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Instructional video 

Instructional materials in the two experiments included two different three-minute video clips about space 
science adapted from Discovery Channel. Each of these two video clips was edited into three versions, i.e., with the 
original music background and sound effects, with Baroque music background and without music. The audio 
narration and volume remained the same through the three treatment versions. 

Research Questions 

The current study had the following questions: 

1. Do the different types of background music 
concepts? 

2. Do the different types of background music 
concepts? 

3. Do the different types of background music 
science concepts? 

Data collection 

Participants were randomly assigned to three groups: (1) instructional video with narration and various 
relevant types of music and sound effects, and (2) instructional video with narration and Baroque music background, 
and (3) instructional video with narration and no background music and sound effects. In each experiment, prior to 
taking the instruction, participants were asked to respond to demographic questions and took a pre-test to ensure 
they were not quite familiar with the space science concepts presented in the video clips. Next, they watched and 
studied the instructional videos. Dependent measures included two post-tests on retention and transfer performance 
and a question on participants’ engagement towards each kinds of treatment. Retention and transfer tests were in 
forms of short answer questions and were scored by the authors. The engagement level was measured by a 5-point 
Likert scale survey. 


and sound effects differ in learners’ retention of space science 
and sound effects differ in learners’ transfer of space science 
and sound effects differ in learners’ engagement level in space 


Results 

Experiment 1 

Table 1. Means and Standard Deviations for Level of Engagement, Retention and Transfer Performance of 
Experiment 1 


Learning 

performance 

Types of background music and sounds 

N 

Mean 

Std. 

Deviation 

Engagement 

Narration + Original music and sounds from 
Discovery Channel 

24 

3.42 

1.060 


Narration + Baroque music 

20 

2.70 

0.932 


Narration only 

24 

2.63 

1.013 

Retention 

performance 

Narration + Original music and sounds from 
Discovery Channel 

24 

4.08 

1.840 


Narration + Baroque music 

20 

4.80 

2.262 


Narration only 

24 

5.17 

2.239 

Transfer 

performance 

Narration + Original music and sounds from 
Discovery Channel 

24 

1.21 

1.351 


Narration + Baroque music 

20 

1.20 

1.508 


Narration only 

24 

1.50 

1.216 


The analysis of variance (ANOVA) tests showed that the effect of background music and sounds on student 
engagement on the instructional material was significant (F(2,67) = 4.442, MSE = 4.487 , p = .016). Post-hoc tests 
showed engagement of the original video of discovery channel is significantly higher than the narration only group 
(p = .022). Meanwhile, there was no significance on retention and transfer performance among the three groups. 
(F(2,67) = 1.625, MSE = 7.258,/? = .205; F(2,67) = .370, MSE = .678,/? = .692). 
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We also found a medium negative correlation between transfer performance and student engagement (r= -.395 ,p = 
.001) and a rather weak negative correlation between retention performance and student engagement (r = -.249 ,p = 
.040). 

Experiment 2 

Table 2. Means and Standard Deviations for Level of Engagement, Retention and Transfer Performance of 
Experiment 2 


Learning 

performance 

Types of background music and sounds 

N 

Mean 

Std. 

Deviation 

Engagement 

Narration + Original music and sounds from 
Discovery Channel 

25 

3.16 

0.987 


Narration + Baroque music 

23 

2.48 

0.947 


Narration only 

26 

2.65 

1.018 

Retention 

performance 

Narration + Original music and sounds from 
Discovery Channel 

25 

6.68 

2.428 


Narration + Baroque music 

23 

6.09 

2.275 


Narration only 

26 

6.81 

2.466 

Transfer 

performance 

Narration + Original music and sounds from 
Discovery Channel 

25 

1.72 

1.242 


Narration + Baroque music 

23 

1.04 

1.186 


Narration only 

26 

2.04 

1.455 


We found that the effect of background music and sounds on student engagement on the instructional 
material was significant (F(2,73) = 3.145, MSE = 3.055, p = .049). Post-hoc tests showed engagement of the original 
video of discovery channel is significantly higher than the baroque music group (p = .05). On the other hand, there 
was a significant difference in transfer performance (F(2,73) = 3.653), MSE = 6.224, p = .031). Transfer 
performance of the narration only group is significantly higher than the baroque music group (p = .026). 

Meanwhile, there was no significance on retention performance among the three groups. (F(2,73) = .617, MSE = 
3.537, p= .543). 

We also found a medium negative correlation between retention performance and student engagement (r= - 
.407, p < .001) and a rather weak negative correlation between transfer performance and student engagement (r = - 
AU, P m .333). 


Discussion and Conclusions 

The results of the correlation prove that there was a negative relationship between level of engagement and 
transfer performance (experiment 1) and retention performance (experiment 2). The results of ANOVA tests and 
post hoc tests in both experiments showed learners in the narration and original sound and music of Discovery 
Channel are more engaged in the instructional video than learners of the other two groups where baroque music and 
no music was used. Nevertheless, learners in the Discovery Channel group learned less than students in the narration 
only group. Even though the results from the two experiments were somewhat inconsistent, they all together suggest 
learners are more engaged when they are learning with interesting sounds and music. This is consistent with arousal 
theory. The results are also consistent with Mayer’s coherence principle, which stated learners would learn better 
without background sounds and music. Suggestopedia’s argument of using baroque music would enhance super- 
learning was not found to be significant in this study. 

The study has a number of limitations. First, the narrators are not the same throughout all experiments. The 
narrator in the narration only group and the narration and baroque music is different from the narrator of Discovery 
Channel, who is a professional voice actor. This may lower the reliability of the instruments used in the study. 
Second, the experiments were to be completed online at students’ own pace. This might have negatively affected the 
quality of responses since participants may get distracted from doing the experiments. We were not able to identify 
the types of music and sounds mixed in the original music used by Discovery Channel; therefore we could not 
embed the same background music and sounds in the condition with narration and baroque music. 

Based on the findings of this study, we would suggest no background sounds and music should be used 
simultaneously with auditory narration. The issue remains is if we do so, learners’ engagement is lowered. Further 
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research on the effects of other different types of music and sounds should be added to increase learners’ 
engagement without trading for learning performance should be done. 
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Effects of Podcasts and Learning Styles on Language Proficiency and 

Attitudes 

Wei Qiang 
James Klein 
Florida State University 

Abstract 

The increased use of podcasts in higher education necessitates research on the impact of this emerging 
technology. While a few studies have been conducted, little empirical evidence exists to support the use of podcasts 
for teaching and learning. The purpose of this paper is to describe a research study conducted to examine the effect 
of podcasting and student learning styles on students’ language proficiency and confidence. The findings indicated 
that listening to grammar podcasts helped students to improve their speaking skills as well as their confidence. 

Introduction 

In the past few years, there has been an increased use of podcasts in higher education. A podcast is “an 
audio [and/or] video content delivery approach based on web syndication protocols” (Abdous, Facer & Yen, 2012, 
p. 44). Students can easily download a podcast and listen to it whenever and wherever they want. 

Advocates of this emerging technology indicate that podcasts are an effective, efficient, and engaging way 
for students to access and review learning materials (Evans, 2007). They believe that podcasting allows students 
control of their own learning which helps them to save time and focus on the important aspects of content (Evans, 
2007). Supporters also claim that podcasts can provide a large amount of authentic materials for foreign language 
learners, which gives them opportunities to learn vocabulary and sentence structures in the real world (Aguilar, 
2007). They also suggest that podcasting is an effective tool for informal and lifelong learning (Aguilar, 2007). 

A few research studies have been conducted recently to examine the use of podcasts for teaching and 
learning. For example, Zanten, Somogyi and Curro (2012) found that students in both face-to-face and distance 
learning settings believe that podcasting is a valuable resource, making them feel less anxious toward learning new 
information. In addition, Abdous et al. (2012) reported that using podcasts as supplementary material in foreign 
language education had a positive impact on learning. 

Aguilar (2007) indicates that a challenge for using podcasts for foreign language learning is that some 
students may not find them engaging or suitable. He suggests that visual learners may not achieve the same learning 
result as auditory learners when they use podcasts. 

The purpose of the current study is to examine the effect of supplementary podcasts and learning styles on 
students’ Chinese language proficiency, and their confidence towards using podcasts for Chinese learning. It aims to 
answer the following questions: 

1 . Does listening to supplementary podcasts help students increase their Chinese listening and speaking skills? 

2. Do auditory learners attain a higher Chinese listening and speaking proficiency level than visual learners 

after using supplementary podcasts? 

3. Does using supplementary podcasts impact students’ confidence? 

Method 

In order to determine the effect of podcasting and learning styles on students’ Chinese proficiency, this 
study employed a quasi-experimental, control group design; intact classes were randomly assigned to either a 
treatment or control group. Both experimental group and control group had the same instructor, course materials, 
teaching strategies and information, and the same assignments. Each week, as part of the assignment for the course, 
the experimental group was told to listen to the selected podcast on grammar and Chinese language usage, which is 
produced almost entirely in Chinese. The control group was told to listen to the selected Chinese culture podcasts, 
which is produced almost entirely in English. Thus, both groups have podcasts but for different content (Chinese 
grammar and usage, Chinese Culture), and in different languages (Chinese, English). 

This study lasted throughout one whole semester. Students’ Chinese listening and speaking proficiency 
were tested before the study as well as after the study. A survey was given to determine students’ learning style at 
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the beginning of the study. Students’ confidence on using podcast for Chinese learning was measured by another 
survey at the end of the study. 

Participants 

Participants were 35 college students enrolled in one of two sections of elementary Chinese II, a course 
offered at a large public university located in the southeastern United States. Before the start of the semester, the 
principal investigator randomly assigned one class section to the experimental group and the other section to the 
control group. All students had completed elementary Chinese I before participating in the current study. There were 
18 male students and 17 female students . The majority of students (80%) were between 19 and 20 years old. 
Students who participated in this study had a variety of academic backgrounds, majoring in Business, International 
Affairs, or Asian Studies. Many (74%) had taken at least one foreign language course before this study, and most 
(91%) took at least one different foreign language course besides Chinese. All participants claimed that they own a 
portable device that can be used to play mp3 files. 

In terms of their prior experience and knowledge of using podcast, 23 students heard about podcasts before 
the study, but only 12 students were subscribing and listening to at least one podcast channel. The top three ways 
they listened to podcasts are: (1) On a computer while studying, (2) While traveling or commuting (on bus, in a car, 
on a bike, or on foot), and (3) During down time (waiting for a ride, between classes, before an appointment). 

Participants were also asked to respond, on a five-point Likert scale (1: strongly agree, 2: agree, 3: 
undecided, 4: disagree, 5: strong disagree), to a thirty-item survey to determine their learning style. The original 
survey, Perceptual Learning Preference Questionnaire, was designed and developed by Reid (1987). This survey 
was designed for non-native English speakers, which aimed to determine their learning style on learning English as 
a foreign language. It was developed based on the existing learning style instruments, and recommendations from 
non-native English speakers and U.S. consultants (Reid, p.92). For the current study, students’ average visual score 
is 34.28, and the average auditory score is 33.60, and there was no significant difference on learning styles between 
the two groups. The descriptive information is listed in the Table 1. 

Table 1 


Descriptive Statistics of students’ learning styles score (N=35) 




Visual 



Auditory 



n 

Mean 

Std. 

Deviation 

n 

Mean 

Std. 

Deviation 

Treatment 

17 

34.47 

5.77 

17 

33.53 

4.33 

Group 

Control 

18 

34.56 

3.20 

18 

33.67 

3.38 

Group 

Total 

35 

34.28 

4.71 

35 

33.60 

3.78 


Procedure 

During Week 1, the study protocol was explained to both class sections and students were asked to sign an 
informed consent form following procedures approved by the university’s human subjects committee. Students were 
told that they would be given access to several supplementary podcasts throughout the semester and were given an 
introduction on how to download and use them. Participants also completed a survey during Week 1. This 
instillment includes three parts - demographic information, prior knowledge and experience related to using 
podcasts, and leaners’ learning styles. During Week 2, all participants were given a pretest to assess their prior 
learning from the elementary Chinese I course. This instalment includes five listening skill items and five speaking 
skill items, and the total points available was 10. In order to track the growth of students’ speaking skills, the 
principal investigator audio recorded their responses. 

Throughout the semester participants were given access to fifteen different podcasts. Each podcast was 
related to the same topic as the lesson content for that week. Students were able to download each podcast and listen 
to it for the whole week. In order to minimize the threat to internal validity, podcasts were assigned to students in 
both the control and the experimental groups each week. The difference was the experimental group used podcasts 
that focused on Chinese language grammar and usage, and mainly hosted in Chinese, while the control group used 
podcasts that focused on Chinese culture and mainly hosted in English. The podcasts used in the study have been 
taken with permission from ChinesePod, a web-based Chinese learning service, which was designed to teach 
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Chinese to people who do not have enough time to take a regular Chinese language course. All participants accessed 
the podcasts via Blackboard. 

At the end of semester, all participants were given a posttest, which used the same format as the pretest; 
Chinese speaking was recorded. Students also completed an online survey to measure their confidence towards 
using podcasts. The survey was modified based on Keller’s IMMS (1987), which is a closed-item Likert style 
questionnaire used to measure four motivational variables related to using podcasts. This current study was only 
focused on the confidence motivational variable. 


Results 

Descriptive statistics were used to compute students’ pretest grades and posttest grades on listening and 
speaking skills. Regression was implemented to examine the predictive relationships between (1) the use of podcasts 
and the posttest grade, and (2) the use of podcasts and the confidence. Correlation statistics were computed to 
determine the relationship between (1) learning styles and posttest grade, and (2) confidence and posttest grade. 

Descriptive Statistics 

The independent-sample t-test results indicated that the participants’ pretest listening grade was normally 
distributed in the treatment group (M=7.86, SD=1.85, N=18) and control group (M=7.15, SD=2.18, N=17), t (33) = 
-.15, p>.05. The independent-sample t-test results also indicated that there is no difference between the treatment 
group (M=7. 1 1, SD=1. 17, N=18) and control group (M=6.68, SD=1.35, N=17) on participants’ speaking pretest 
grade, t (33)= -.10, p>.05. Table 2 also showed the descriptive statistics on participants’ pretest and posttest listening 
and speaking grades. There were 1 7 participants in the control group and 1 8 participants in the treatment group 
throughout the study. The average scores for each test are: 7.50 (pretest listening score), 7.06 (pretest speaking 
score), 6.85 (posttest listening score), and 7.76 (posttest speaking score). This indicated that participants’ speaking 
score was increased after the intervention. 


Table 2 

Descriptive Statistics of students’ Pretest and Posttest score (N=35) 




Pretest 


Posttest 



Listening 

Speaking 

Listening 

Speaking 

Control 

Mean 

7.15 

6.68 

6.68 

7.38 

group 

Std. Deviation 

2.18 

1.35 

1.07 

1.14 

Experiment 

Mean 

7.86 

7.11 

7.03 

8.14 

group 

Std. Deviation 

1.85 

1.17 

1.02 

1.05 

Total 

Mean 

7.50 

7.06 

6.85 

7.76 

Std. Deviation 

2.02 

1.53 

1.05 

1.15 

Regression Analysis 


Table 3 shows the regression analysis data on using podcasts as predictor to predict participants’ posttest 
speaking score. The results indicated using podcast as supplementary materials significantly impacted posttest- 
speaking score between the treatment group (grammar) and control group (culture), F (1, 33)=4.16, p< .05, 
explaining 1 1.2% of the variance. (P = .76, p< .05). 

Table 4 shows the regression analysis on using posttest as predictor to predict participants’ confidence on 
using podcast to learn Chinese. The average students’ confidence score is 22.61. Results revealed a significant 
difference between the treatment group and control group, F (1, 33)= 5.95, p< .05, explaining 15.3% of the variance 
(P = .4.34, p< .05). Students who received grammar podcasts had higher confidence than those who received culture 
podcasts. 

Table 3 

Usin g the treatment to predict posttest-speaking score 



Predictor 

P 

Sig. 

1 

Grammar 

.757 

.049 

R 2 


.112 


Model F 


4.163 



168 



Table 4 

Usi ng the treatment to predict confidence sc ore 



Predictor 

P 

Sig. 

1 

Grammar 

4.34 

.020 

R 2 


.153 


Model F 


5.95 



Correlation analysis 

The correlation analyses were run to examine whether there was a significant relationship between learning 
styles and posttest scores. The results showed that there was a significant positive relationship between the auditory 
score and students’ posttest listening score, r (33)= .28, p< .05 (one-tailed). There was also a significant negative 
relationship between the visual score and students’ posttest speaking score, r (33)= -.32, p< .05(one-tailed). Another 
positive correlation was found between confidence and posttest speaking test score, r (33)= .049, p< .001. 

Discussion 

The current study was designed to examine the effect of podcasts and learning styles on students’ Chinese 
speaking and listening skills. The descriptive statistics showed that students’ speaking scores were increased after 
the intervention, and the further regression analysis showed that students who received grammar podcast performed 
better on their speaking test than students who received culture podcast. This might be because students who 
received grammar podcasts had more comprehensive language input, which helped them to better understand the 
content. Then with the better understanding of the grammar in their minds, students produced the language better. 

Regression analysis also indicated that students who received grammar podcasts felt more confident to 
learn Chinese than students who received culture podcasts. This is because for the grammar group, the content was 
about how to use particular grammar in real life, and the grammar was also aligned with the course content. Thus, 
students received more explanation and examples on how to use the language, so they felt more confident to learn 
Chinese. On the other hand, for the control group, their content was about Chinese culture. While culture is 
important in language learning, the podcast content did not directly relate to how to use Chinese. Therefore, for the 
instructor who wants to use podcasts as supplementary materials, the initial question to ask is “what do you want the 
podcast to do for your students?” Thus, the podcast content can directly relate to the learning outcome. 

Another interesting finding was that there was a significant correlation between students’ confidence and 
their posttest speaking score. This meant that students who receive podcasts that directly relate to the learning 
outcome as extra help can increase their confidence, which may explain why they also had a high speaking score. 

Conclusion 

The use of emerging technology for teaching and learning requires that educational technology researchers 
collect data on their efficacy. The current study provides empirical evidence on the use of supplementary podcasts 
for students to improve their foreign language skills despite different learning styles. It also shows that podcasts 
have a positive effect on students’ confidence when learning a foreign language. 

Reference 

Abdous, M., Facer, B. R., & Yen, C.J. (2012). Academic effectiveness of podcasting: A comparative study of 

integrated versus supplementary use of podcasting in second language classes. Computer & Education, 58, 
43-52. 

Aguilar, F. R. (2007). Top of the pods-in search of a podcasting “podgogy” for language learning. Computer 
Assisted Language Learning, 20 (5), 471-492. 

ChinesePod. (2012). ChinesePod Ltd. Retrieved from http://chinesepod.com/about . 

Evans, C. (2007). The effectiveness of m-learning in the form of podcast revision lectures in higher education. 
Computer & Education, 50, 491- 498. 

Keller, J. M. (1987). IMMS: Instructional materials motivation survey: Florida State University. 

Reid, J. M. (1987). The learning style preferences ofESL students. TESOL Quarterly, 27(1), 87-111. 

Zanten, R.V., Somogyi, S., & Curro G. (2012). Purpose and preference in educational podcasting. British Journal of 
Educational Technology’, 43 (1), 130-138. 


169 



Gender and Gaming: A Literature Review 

Danae Romrell 
Idaho State University 
643 Meadowbrook 
Rexburg, ID 83440 
romrdana@isu.edu 


Descriptors: Gender, Gaming 


Abstract 

Educational games are commonly used to help motivate students and provide enhanced learning 
opportunities. However, because of gender differences in how males and females play games, concerns 
regarding potential gender inequity arise when games are used for education. In order to explore what 
gender differences exist and how they might impact educational games, this literature review summarizes 
the common themes and findings of recent research articles focusing on gender and gaming. In particular, 
this review discusses the demographics of video game players, especially as they relate to gender; 
characteristics of female game players; gender differences in motivation and game style preference; gender 
and educational games; and the implications of gender differences on video game design. 

Introduction 

As technology becomes more prevalent in education, students of all ages have increased 
opportunities to play and learn from educational games. At first glance, educational games appear to be an 
ideal educational tool because they provide increased motivation for students to spend time engaged with 
academic concepts. However, concern arises as possible gender differences in gaming are considered. 
Stereotypically, most game players are males (Shaw, 2012) and females typically exhibit higher levels of 
anxiety than males with regards to playing online games (Huang, Hood, & Yoo, 2013). If not appropriately 
addressed, gender differences could promote gender inequity when games are used for education. 

The topic of gender differences in video game play has been addressed often in recent research. 
The purpose of this literature review is to explore that recent research to determine if there are any gender 
differences in how people interact with games. Is so, what are those differences? And what impact should 
those differences have on game design? 

Although gender differences in educational games is the desired focus of this research, there is 
much that can be learned by examining gender differences in gaming for entertainment. As such, this 
literature review will look at research on all types of games. This includes games played on all types of 
media such as computers, handheld devices, and dedicated gaming consoles. 

Who Plays Video Games? 

Most researchers seem to agree that the stereotypical video game player is a young, white, 
heterosexual male (e.g. Shaw, 2012; Williams, Consalvo, Caplan, & Yee, 2009). In considering the topic of 
gender and gaming, one of the first questions to arise is whether or not this stereotypical view of video 
game players is accurate. Each year the Entertainment Software Association (ESA) publishes a report 
containing statistics on video game player demographics. Many of the articles selected for inclusion in this 
literature review cited gender statistics from the ESA annual report (e.g. Shaw, 2012; Williams Consalvo, 
et al., 2009). Statistics from five ESA annual reports are summarized in Table 1. As these results are 
considered over time, it appears that although more men than women play video games, the percentage of 
women game players is increasing over time. However, as these statistics are evaluated, it is important to 
realize that the ESA uses a very broad definition of what constitutes a video game. Any digital game is 
included in this statistic, including games played on smartphones, tablets, or other handheld devices 
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(Entertainment Software Association, 2012). Two other studies included in this literature review looked at 
game player demographics. Williams and Consalvo, et al., (2009) found that 80.22% of Everquest II 
players were male. Similarly, Yee (2006) found that 85.4% of players of massively multiplayer online 
role-playing games (MMORPG) were male. From these studies we see that although the most recent ESA 
annual report shows that approximately 45% of game players are female (ESA, 2013), the percentage of 
female players varies drastically depending on the type of game. 


Table 1 

Summary of Research on Gender Demographics for Video Game Players 


Source 

Data Based On 

Percentage Male 

Percentage 




Female 

Entertainment Software 
Association (ESA, 2006) 

2006 ESA Annual Report 

62% 

38% 

Entertainment Software 
Association (ESA, 2008) 

2008 ESA Annual Report 

60% 

40% 

Entertainment Software 
Association (ESA, 2011) 

2011 ESA Annual Report 

58% 

42% 

Entertainment Software 
Association (ESA, 2012) 

2012 ESA Annual Report 

57% 

43% 

Entertainment Software 
Association (ESA, 2013) 

2013 ESA Annual Report 

55% 

45% 

Williams, Consalvo, et al., (2009) 

Random sample of Everquest II 
players (mean age = 33) 

80.22% 

19.72% 


Survey data from 30,000 MMORPG 



Yee (2006) 

users 

85.4% 

14.6% 


(Mean age = 27) 




Although it is common for both men and women to say that they play games, it is less common for 
individuals of either gender to describe themselves as a gamer. In particular, women are less likely than 
men to self-identify as a gamer (Shaw, 2012). Women are also more likely than men to underrepresent the 
amount of time they spend playing video games (Shaw, 2012; Williams, Consalvo, et al., 2009). By 
comparing self-reported data to data collected from game servers, Williams and Consalvo et al. found that 
women underrepresented their playing time by an average of 3 hours per week, while men 
underrepresented their playing time by an average of 1 hour per week. Overall, men have more positive 
attitudes about gaming than women (Bonanno & Kommers, 2008). 

Another interesting consideration related to gender demographics in games is the gender of game 
characters. Williams, Martin, Consalvo, and Ivory (2009) used a sample of 133 different video game titles 
to examine the gender distribution of game characters. Their results are summarized in Table 2. Notice 
that most of the game characters are male, especially if only primary game characters are considered. This 
indicates that many of the female characters in video games have a secondary role. The authors reported 
two statistics for the distribution of male and female characters, a weighted percentage and an unweighted 
percentage. The unweighted percentage considers all game characters equally while the weighted 
percentage weights characters based on the number of game copies sold. Notice that the statistic that is 
weighted has a higher percentage of male characters. This indicates that although game designers do create 
some games with female characters, the games that are being purchased are biased in favor of male game 
characters. 

Based on these studies, we see that some women do play video games, although not as prevalently 
as men. In fact, women who do play games actually play games more often and for longer periods of time 
than men. The group of players who plays at the highest rates is older female players (Williams, Caplan, & 
Yee, 2008; Williams, Consalvo, et al., 2009). In order to gain a better understanding of these women, we 
now turn to literature that considers women as gamers. 
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Table 2 


Gender Distribution for Game Characters (Williams, Martin, et al., 2008) 


Character Group 

Percentage Male 

Percentage Female 

All Characters (weighted) 

85.2% 

14.7% 

All Characters (unweighted) 

81.2% 

18.3% 

Primary Characters 

90% 

10% 


Women as Gamers 

Two of the studies selected for inclusion in this literature review explored the game play 
characteristics of female game players. In order to illustrate their findings, these two articles will be 
described. The first study was conducted by Hayes (2007) and used a qualitative case study research design 
to describe the game play experiences of two women. The second study by Royse, Lee, Undrahbuyan, 
Hopson, & Consalvo (2007) used data obtained from focus groups and in-depth interviews with women to 
describe the game play characteristics of women game players at different levels of play. 

Hayes (2007) detailed the experiences of Joanna and Deirdre, two female graduate students who 
were required to play an online role-playing game as a requirement for a graduate level course on video 
games and learning. With both women, several aspects of their game play can be seen as a reflection of 
their life experiences and personalities. 

For example, consider the following connections between Joanna’s personal background and her 
game play. Joanna grew up in a family of academics who pushed her to do well in school. Joanna had the 
academic talent to succeed, but described herself as a troublemaker who lacked the motivation to do well in 
school. As a result she barely graduated from high school. Joanna’s self-described identity as a 
troublemaker was evident in her game play. For example, while playing the game she stole items from 
other players and developed a reputation as a thief. She also tried to work around the rules of the game by 
using cheat codes she found online to avoid the aspects of the game she was less interested in. 

Deirdre’s personal background also showed up in her game play choices. Deirdre was very 
involved in a church youth group and was passionate about helping others. She had excelled in science in 
school, especially in chemistry. This combination of being good at science and her passion for helping 
others led her to consider a career as a physician. This background appeared in her game play as Deirdre 
made game character choices that allowed her to be a lay-healer and a magician. As a magician she created 
potions, something she said she liked because it reminded her of chemistry. 

These two examples, and several other detailed by Hayes (2007), provide support for Hayes’ 
conclusion that the personal background, personality, and life experiences of women have an impact on 
their game play. Hayes argues in favor of “developing a more holistic approach to understanding women’s 
choices, pleasures, and challenges in game play, one that takes into account their past, present, and 
anticipated identities and goals” (p. 42). This case study research provides an initial look at how women’s 
identities affect their game play. However, because of the small scale of the qualitative study, further 
research would be needed in order to generalize the results or draw any broad conclusions. 

In her study, Royse et al (2007) separated the female participants into three separate groups: power 
gamers, moderate gamers, and non-gamers. Power gamers were defined as those who played games for 
more than three hours each week, moderate gamers played approximately one to three hours per week, and 
non-gamers did not play any video games. Table 3 summarizes some of the differences found among the 
three groups. Of particular interest is the difference in players’ opinions of violent content and the over- 
sexualization of female characters. The power gamers were not concerned with these issues while the 
moderate gamers tended to avoid very violent games and expressed concern about the roles of female game 
characters. Non-gamers had very negative, critical opinions about games in general, for these and other 
reasons. 

This difference of opinion among women with differing levels of experience with game play 
seems to support the possibility suggested by Hayes (2007) that gender might be confounded with 
inexperience in video game research. Women in the Royce et al. (2007) study who were experienced game 
players did not express the same opinions and concerns as those who played games less often or not at all. 
Hartmann and Klimmt (2006) further support this possibility with their observation that inexperienced male 
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game players often had game preferences that more closely matched those of women than those of 
experienced male players. 


Table 3 

Characteristics of Female Gamers at Different Levels of Game Play (Royce et al., 2007) 


Characteristic 

Power Gamers 

Moderate Gamers 

Non-Gamers 

Level of Game Play 

Played more than 3 hours 
per week 

Played about 1-3 hours 
per week 

Did not play games 

Reasons for Game Play 

Technologically advanced 
players who both played 
and enjoyed a variety of 
games 

Played games to avoid 
pressures of everyday life 

Consciously chose not to 
play games and had 
negative, critical opinions 
of games and gamers 

Opinions about combat 
and violent content 

Enjoyed competition and 
combat 

Typically avoided the 
very violent games 

Worried about game 
violence and addiction 

Opinions about game 

Wanted game characters 

Saw themselves as 

Asserted their own 

characters or game 
players 

that were sexy and strong 
Enjoyed beating male 
players 

controlling the character, 
not being the character 
Expressed concern about 
over-sexualization of 
characters. 

interpersonal skills while 
disparaging the perceived 
asocial behavior and lack 
of interpersonal skills of 
gamers 


Gender Differences in Motivation for Game Play 

One main gender difference that emerged in the research is motivation for game play. A series of 
three related articles were particularly helpful in examining gender differences in game play motivation. 

The first study by Yee (2006) was later expanded and replicated in a second study by Williams et al. 

(2008). Additional research reported by Williams, Consalvo, et al., (2009) in a third article further 
expanded the results reported in the second article. 

In the first article in this series, Yee (2006) used survey data from 6,675 players of massively 
multiplayer online role-playing games (MMORPG) to research game play motivation. Using an exploratory 
factor analysis, he found five factors that describe the motivations of players: relationship, manipulation, 
immersion, escapism, and achievement. For each of the five factors, Yee found a significant difference 
between the scores for male and female game players. In particular, males were more likely to be 


Table 4 

Descriptions of Game Motivation Categories (Williams et al., 2008) 


Motivation 

Examples 

Achievement 

• 

Advancement 


• 

Analyzing Game Mechanics 


• 

Competition 

Social 

• 

Chatting 


• 

Casual Interactions 


• 

Developing supportive relationships 


• 

Teamwork 

Immersion 

• 

Geographic exploration 


• 

Role-playing 


• 

Avatar Customization 


• 

Escapism 
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motivated by achievement or manipulation, while females scored higher on relationship, immersion, and 
escapism. 

Williams et al. (2008) used Yee’s (2006) results to develop questions for a survey of players of the 
MMORPG game Everquest II. This follow-up study used a factor analysis to further revise the motivations 
for game play. The authors identified three main motivational factors: achievement, social, and immersion 
(see Table 4). Using the same survey results and data set, Williams, Consalvo, et al. (2009) looked for 
gender differences among these three motivation categories. They found that males were significantly 
more motivated by achievement factors, women were significantly more motivated by social factors, and 
there was no gender difference in motivation by immersion factors. These results largely replicated the 
earlier results found by Yee. 

Overall, this series of articles contributed greatly to the existing research on motivations for game 
play. The data used in the latter two studies is particularly valuable because survey data from Everquest II 
players was linked to actual game play data provided by the Everquest II game servers. This led to data 
that was more reliable than the self-reported data used by most of the other studies included in this 
literature review. Additionally, the series of three articles provided replication for the same results based on 
two different sets of data. One weakness of the series of articles is that they apply to only one type of game. 
Games that are not MMORPGs were not accounted for in the studies, so the results cannot be generalized 
to all game play. However, this gap may be somewhat filled by a study by Hartmann and Klimmt (2006) 
that also addressed motivation for game play. This study used survey data and included game players who 
played a wide variety of video game types. Their results were similar to those found in the series of articles. 
Specifically, Hartmann and Klimmt found that men are more likely to play games because they enjoy 
competition and have a need to win. Women are more likely to play because they enjoy the social 
interaction provided by game play. 

From these research studies it seems clear that men and women have different motivations for 
playing games. Generally, men are more motivated by achievement and competition while the women are 
more motivated by the social aspects of the game. Given that men and women have different motivations 
for game play, it would be reasonable to hypothesize that they enjoy playing different kinds of games. 

Gender Differences in Game Style Preference 

Several of the articles included in this literature review addressed the topic of game style 
preference (see Table 5). It is somewhat difficult to compare the results of these studies based on the very 
different descriptive labels used by researchers for game styles. However, even though different names 
were used, a general pattern of game style preferences based on gender does emerge. 

Table 5 

Summary’ of Research on Game Style Preferences 


Researcher 

Age of Subjects 

Male Game Style 
Preference 

Female Game Style Preference 

Lowrie & Jogensen 
(2011) 

10-12 years old 

Action Games 

“Other” Games (e.g. Brain 
Training, Buzz!) 

Quaiser-Pohl, Geiser, & 
Lehmann, (2006) 

High School Students 
(Mean age =14.9) 

Action and 
Simulation Games 

Non-players or 
Logic/Skill Training Games 

Karakus, Inal, & 
Cagiltay, (2008) 

High School Students 
(Grades 9-12) 

Car Race 
Sports Games 
First Person Shooter 

Action Adventure 
Puzzle Games 
Car Race 
Card Games 
Board Games 
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Kinzie & Joseph (2008) 

Middle School 
Students 
(Mean age = 12) 

Active 

Strategic 

Creative 

Explorative 

Hartmann & Klimmt 
(2006) 

Adults 

(ages 18-26) 

Competitive 

Games with little violence, a non- 
sexualized role for females, and 
opportunities for social 
interaction 


In describing the game style preference of individuals in their study, Kinzie and Joseph (2008) 
advocated using descriptions of the activity modes available in a game rather than using more traditional 
game style descriptions (e.g. first person shooter or action game). They define an activity mode as a 
description of a type of activity that game players like to engage in as they play video games. Six activity 
modes were identified: active, explorative, problem-solving, strategic, social, and creative. Using a survey 
of 42 middle-school age children, they determined that more children than expected prefer to engage in 
exploratory mode and fewer children than expected prefer to engage in problem-solving or social mode. 

As they examined gender differences the researchers found that males were more likely to prefer active or 
strategic mode while females were more likely to prefer exploratory or creative mode. 

Quaiser-Pohl, Geiser, and Lehmann (2006) used a sample of 861 high school age children to 
study different types of computer game players. Using a latent class analysis (LCA) they divided game 
players from their sample into three different groups. The three groups, called latent classes, grouped 
together subjects who expressed similar game -playing preferences on a computer game questionnaire. 

Table 6 summarizes the game playing preferences of the three latent classes. Note that gender was not a 
consideration used creating the classes in the LCA. However, after completing the LCA the authors 
determined the gender distribution within each latent class. Interestingly, Class 1 was made up of 
approximately 82% females, Class 2 was made up of approximately 82% males, and Class 3 was made up 
of approximately 83% females. Using a chi-square test for independence, the authors determined that there 
is a strong association between computer game style preference and gender. 

Table 6 

Gender distributions of latent class profiles grouping subjects with similar game playing preferences (Quiaser- 
Pohl et al„ 2006) 


Latent Class 

Description 

Gender Distribution 

Class 1 

Not likely to play 

18% male; 82% female 

Class 2 

Most likely to play action or simulation 
games 

82% male; 18% female 

Class 3 

Most likely to play logic or skill training 
games 

17% male; 83% female 


Karakus, Inal, and Cagiltay (2008) looked both at what types of games children like to play and 
why they select that particular type of game. Their results were based on a sample of 1224 high school age 
children from eight different high schools in Turkey and found that males generally preferred to play games 
that were classified as car race, sports, or first person shooter games. Females preferred to play games that 
were classified as action adventure, puzzle, car race, card, or board games. Males were more likely to select 
a particular game because it was entertaining and females were more likely to select a particular game 
because it was instinctive. One weakness of this study was that although some of the questions on the 
survey were open-ended, many of them were multiple choice and contained a very limited number of 
options from which the student could select their answer. In many cases the choices were rather arbitrary 
and did not cover all possible answers. 

Lowrie & Jogensen (2011) also conducted a study to determine if there were any differences in the 
types of games that children like to play. Their study specifically focused on children between the ages of 
10 and 12. They determined that males were more likely to play action games and females were more likely 
to play games labeled as ‘other’ (the other category contained games such as BrainTraining and Buzz!). 
They found no significant difference in simulation games, which was rated highly by both males and 
females. 
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Hartmann & Klimmt (2006) took a slightly different approach to examining game style 
preference. Rather than looking at the type of game (e.g. first person shooter or puzzle game), they looked 
at the type of content that was preferred by women. They created five fictional game descriptions that 
appeared to be the back of a video game package and asked the females in their study which games they 
would like to play. They found that the preference was to play games that had less violence, a non- 
sexualized role for female characters, and lots of opportunities for social interaction. Because this study 
was conducted with only female subjects, we are not able to determine whether there are any gender 
differences based on this result. A second study reported in the same article included both male and female 
subjects and found that men were more likely than women to play highly competitive games. A similar 
gender difference was not found in non-competitive games. 

Looking collectively at these five separate studies (see Table 4) highlights the gender difference in 
game style preference. All four studies found very similar results, although they may have used different 
terms to describe the games. It appears that males prefer games with a lot of action and competition, while 
women tend to prefer logic, puzzle, or skill training games. 

The game style preference for males also seems to align with the game play motivations discussed 
in the previous section of this literature review. Recall that men were motivated by achievement, which 
may be one reason why they enjoy playing games with lots of action and competition. For females, the 
same alignment is not universally seen in all of the studies. Although, most of the research agreed that 
females were motivated by social interaction, not all of the articles on game style preference found social 
games to be preferable to women. Hartmann & Klimmt (2006) did find that women preferred games with a 
lot of opportunities for social interaction. However, Karakus et al. (2008) found that only 1.7% of females 
and 2.2% of males selected games because of the social environment of the game. Kinzie and Joseph 
(2008) found that overall their subjects had a “lesser desire for game play involving. . .social modes” (p. 
654) and that there was no gender difference in preferences for social modes. One reason for this seemingly 
contradictory research may be that all of the studies on motivations for game play were conducted with 
adults playing games for entertainment purposes and most of the studies on game style preference were 
conducted with middle- or high-school age children. As such, the results really don’t apply to the same 
population of game players. 


Gender and Educational Games 

Given that males and females have different game play motivation and game style preferences, 
educators should question whether or not educational games provide equal learning opportunities to both 
males and females. In order to help address this issue, Papastergio (2009) conducted a study comparing 
learning from an educational game and learning from a web-based application in order to determine if there 
was a gender difference. The high school students who participated in the study were divided into two 
groups. In group A the content was delivered through an educational game. In group B the same content 
was delivered through a web application. Initially, the researcher cited gender differences in gaming as the 
basis for his hypothesis that males would have greater learning gains from the educational game and 
females would have greater learning gains from the web application. However, the results of the study did 
not support this hypothesis. The only gender difference in the study was in pretest scores. Although 
females had lower pretest scores than males, there was no significant gender difference in learning gains for 
the educational game or for the web application. Both males and females exhibited significantly higher 
learning gains when content was delivered through the educational game than they did when content was 
delivered using the web application. Papastergio claims that this study provides evidence that educational 
games are a valuable resource for helping students learn and they work equally well for both genders. One 
weakness of this study is its limited scope. It dealt with students in one class on one subject. There is a 
need for similar results to be shown for other populations in different subjects and age groups. 

If the results of Papstertio’s (2009) study hold true among other populations and educational 
games provide greater learning gains for both males and females than other web applications, then it will be 
especially important to consider how to best design educational games. Ideally, the goal would be to create 
“educational games that children will choose to play in their free time” (Kinzie & Jospeh, 2008, p. 661). 
Educational game designers are trying to create educational games that students find fun and enjoy playing, 
but that also help the student engage with educational content in a meaningful way. Research on 
educational games has identified a few roadblocks to this goal. 
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Children often find educational games to be uninteresting. Boys in particular find educational 
games to be boring (Kinzie & Joseph, 2008). Girls expect games to have an instructional component while 
boys just want games that are entertaining and competitive (Karakus et al., 2008). Bonnano & Kommers 
(2008) found that females often see games as just one possible way to learn something that could also be 
learned another way. Males, on the other hand, see games as a unique learning experience that couldn’t be 
achieved in another way. This tends to make females more skeptical about the value of games for 
education than males. In order to overcome these potential game design difficulties, it becomes very 
important that educational games are well designed using a method that takes gender differences into 
account. 

Implications of Gender on Game Design 

As has been detailed in earlier sections of this literature review, there are gender differences in 
motivations for game play and game style preference. In order to appeal to a larger audience, game 
designers should come to a better understanding of those differences. Some specific design suggestions 
were provided in a few of the articles included in this review. Specifically, Kinzie and Joseph (2008) offer 
some suggestions for designers of educational games and Shaw (2012) provides suggestions for designers 
of games for entertainment. 

Kinzie and Joseph (2008) base their suggestions for game designers on the results of their study of 
activity modes in video games. They suggest incorporating opportunities for students to engage in 
exploratory mode or problem-solving mode to appeal to players of both genders. Exploratory mode is 
particularly desirable because it scored quite high for both males and females, but it scored especially high 
for females who might already be at a disadvantage when it comes to video games because of the cultural 
and social idea that video games are better for boys. In designing a game that is meant to appeal 
particularly to girls, such as a game to help girls develop an interest in math and science, Kinzie and Joseph 
recommend using the creative activity mode. And to help promote interest in educational games for boys, 
who are often bored by educational games that they don’t find fun, they recommend providing 
opportunities for active and strategic modes. Although they provide suggestions for creating games that 
appeal to a single-gender, Kinzie and Joseph’s main recommendation is to incorporate as many activity 
modes as possible into an educational game in order to appeal to as broad of a game playing audience as 
possible. 

Shaw (2012) points out that by creating ‘girl games’, game designers increase the gender divide in 
the gaming industry by further separating women game players from the more mainstream games. Rather 
than creating games that appeal to a single target audience, Shaw (2012) recommends that designers create 
games that incorporate diversity. 

It seems particularly important to notice that researchers provide the same suggestion to designers 
of both educational games and games for entertainment: create games that appeal to a diverse game 
playing audience. Progress towards achieving this goal might also be made if there were more female 
game designers. Williams, Martins, et al. (2009) found that the demographics of video game designers 
closely mirror the demographics of game characters. In particular, approximately 80% of game designers 
are male and about 80% of game characters are male. Having both male and female game designers 
working together on the creation of games may help achieve the goal of designing games that appeal to a 
broad game playing audience. 

Conclusion 

Based on the articles included in this literature review, there do appear to be gender differences in 
how people interact with video games. In particular, men and women have different motivations for game 
play and different game style preferences. 

Men generally prefer to play games that are active and competitive, while women prefer logic, 
puzzle, and skill training games and enjoy social interactions provided by games. These gender differences 
provide some unique challenges to game designers as they attempt to design games that appeal to as broad 
of a game playing audience as possible. 

Games are often used in educational settings because it is thought they provide increased learner 
motivation. It is particularly important in an educational setting to use games that appeal to both genders in 
order to provide equal educational opportunities to all students. Based on the recommendations of Kinzie 
and Joseph (2008), designers of educational games should try to incorporate as many different types of 
game play activity as possible. In particular, opportunities for exploratory play are particular attractive to 
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children of both genders. As game designers incorporate these and other game design suggestions in order 
to appeal to children of both genders, they will hopefully come closer to achieving the goal of creating 
games that will allow all children opportunities to learn. 
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Appendix A 


Appendix A 

Brief Summary’ of Articles Included in the Literature Review 


Researchers 


Bonnano & 
Kommers (2008) 


Methodology Problem Statement, 

and/or Subjects Research Question(s), or 

Hypothesis(es) Related to 
Gender 

Development of an Is there a gender difference 

instmment to in attitudes toward gaming? 

examine gaming 
attitudes. 

Survey of 170 
students (ages 16-18) 


Summary of Findings 


Males are more confident in 
game playing skills and game 
hardware than females. 

Males played for 
entertainment and relaxation 
while females were more 
likely to only play games for 
education. 


Overall males have a very 
positive attitude about 
gaming and females have a 
less positive or neutral 
attitude towards gaming. 


Hartmann & 
Klimmt (2006) 


The first study was 
based on a sample of 
317 females (ages 
18-26). 

The second study 
used an online 
survey of 795 users 
of German 
computer-game- 
related websites 
(only 18 participants 
were female). 


The first study hypothesized 
that females interest would 
be low if a video game 
involved violence, an over- 
sexualized female 
protagonist, and few 
opportunities for social 
interaction. 

The second study 
hypothesized that more men 
would play competitive 
computer games than 
women and that men would 
express more of a need to 
win. 


Women typically prefer 
games with low violence, lots 
of social interaction, and a 
non-sexualized portrayal of 
women. However, there were 
women who went again the 
general trend, most often with 
respect to violence. 

Men were much more likely 
than females to play highly 
competitive games. 


Hayes (2007) 


Case Study Research 

Two female adult 
graduate students 


Describe the experiences of 
two women with playing 
Morrowind. Explore how 
they enacted gendered 
identities and how they 
engaged with new forms of 
identity. 


The two women both let their 
own personal identity impact 
the choices they made, more 
than just making choices 
because they were women. 

Women from similar 
backgrounds can have very 
different responses to 
gendered practices. 
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Researchers 

Methodology 
and/or Subjects 

Problem Statement, 
Research Question(s), or 
Hypothesis(es) Related to 
Gender 

Summary of Findings 

Karakus, Inal, 
Cagiltay (2008) 

Descriptive Study 

Survey of 1224 
high school age 
children in Turkey 

What are the most favored 
computer game genres for 
males and females? 

What aspects of those 
games do they like the 
most? 

What is their opinion of 
the value of games for 
education and the effects 
of games on players’ 
behavior? 

Males preferred car race, sports, 
and first person shooter games. 
Females preferred action adventure, 
puzzle, car race, card, or board 
games. 

Females wanted games with an 
instructive element while males 
wanted games that were 
competitive and entertaining. 
Females were more skeptical about 
the value of games for education 
and expressed more concern about 
negative behavior effects of games. 

Kinzie & 

Survey of 42 

The survey attempts to 

Children prefer to play with a 

Joseph (2008) 

middle school age 
children (average 
age is 12) 

describe information in 
four categories: 1) 
demographics and game 
play experience, 2) 
character, setting, and help 
preferences, 3) activity 
mode preference, and 4) 
activity mode attitude. 

character that is similar to 
themselves in gender and ethnicity, 
but slightly older. The most 
preferred activity modes for boys 
were strategic and active. For girls, 
creative and explorative. There 
was no gender difference for the 
problem-solving or social activity 
modes. 

Lowrie & 

Survey of 426 

Are there gender 

Boys played games more days per 

Jorgensen 

children in 

differences in the amount 

week than girls. On school days 

(2011) 

Australia (ages 10 - 
12) 

of time spent playing 
games? 

Are there gender 
differences in the types of 
games children play or the 
mathematics sense-making 
obtained through games? 

there was not a significant 
difference in the amount of time 
spent playing games. On non- 
school days, boy played 
significantly more that girls. 

Boys liked games with more action 
and girls liked the “other” category. 
Both liked simulation games the 
most. 

Papastergiou 

(2009) 

Experiment 

88 Greek high 
school students in a 
required computer 
course 

Hypothesized that males 
would exhibit greater 
achievement and have 
more positive attitudes 
than females when 
learning was based on an 
educational game. 

Hypothesized there would 
be no gender difference 
when learning was based 
on a web-based non- 
gaming application. 

They found no gender differences 
in achievement or in attitudes. 

Both boys and girls had greater 
achievement and more positive 
attitude when learning was based 
on an educational game rather than 
a web-based non-gaming 
application. 
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Researchers 

Methodology 
and/or Subjects 

Problem Statement, 
Research Question(s), or 
Hypothesis(es) Related to 
Gender 

Summary of Findings 

Quaiser-Pohl, 
Geiser, & 
Lehmann 
(2006) 

Used a mental 
rotation test (MRT) 
and a computer 
game experience 
questionnaire. 

861 individuals 
(ages 10-20) 

They hypothesized that 
males and females would 
have different game 
preferences and that males 
would perform better than 
females on the MRT. 

There is a gender difference in 
game preference with males being 
more likely to play action or 
simulation games and females 
being more likely to not play any 
games or to play logic or skill 
training games. 

Males performed better than 
females on the MRT. Males who 
played games often performed 
better on the MRT than males who 
did not play as often. 

Royse et al. 
(2007) 

Qualitative research 
to provide thick 
descriptions of 
women game 
players 

Focus groups and 
interviews with 20 
female adults 

Why do women play 
games? 

How do women’s 
perceptions of themselves 
affect their decision to play 
games? 

Do games have an effect 
on women’s identity? 

Females who with different levels 
of game play experience played 
games for very different reasons 
and had very different opinions 
about games and gamers. 

Findings support the theory that 
“gender and technology have a 
reciprocal relationship” (p. 574). 

Shaw (2012) 

Ethnographic and 
Grounded Theory 
Research 

Interviews with 27 
adults 

Determine how and why 
people identify as gamers 
or not. 

Men are more likely than females 
to identify as a gamer. 

There is a perceived negative 
connotation for identifying as a 
gamer. 

Williams, Yee, 
& Caplan 
(2008) 

Descriptive Study/ 
MANOVA 

Survey of 7,129 
adult Everquest II 
(EQII) game players 

What is the gender 
distribution of MMO 
players? 

What are the motivations 
of MMO players? 

Men outnumbered women 4 to 1 , 
but that the average playing time 
was higher for women and for 
older gamers than it was for young 
players or male players. Older 
female players are playing at the 
highest rates. 




Motivations for game play can be 
divided into three categories: 
achievement, social, and 
immersion. 

Williams, 
Consalvo, 
Caplan, & Yee 
(2009) 

Descriptive Study/ 
MANOVA 

Survey of 7,129 
adult Everquest II 
(EQII) game players 

What are demographic 
differences between men 
and women? 

What gender differences 
are there for motivation, 
length of game play, and 
health benefits? 

Men are more motivated to play by 
achievement; women are more 
motivated to play by social 
interactions. 

More men play games, but women 
who play are older and play for 
more hours than men. 
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Researchers 

Methodology 
and/or Subjects 

Problem Statement, 
Research Question(s), or 
Hypothesis(es) Related to 
Gender 

Summary of Findings 

Williams, 
Martin, 
Consalvo, & 
Ivory (2009) 

Sample of 133 
different video 
game titles - 
weighted by 
popularity 

How frequently are males 
and females represented as 
characters in video games? 

Most video game characters (85%) 
are male. This percentage of male 
characters is even higher (90%) 
when considering only primary 
characters. 




The percentage of male characters 
is higher when games titles are 
weighted by popularity - indicating 
that games with male characters 
are purchased more than games 
with female characters. 

Yee (2006) 

Online survey of 
30,000 MMORPG 
players over a 3- 
year period 

What are the demographics 
of players of MMORPG? 

What are game play 
motivations for MMORPG 
players? 

Includes a detailed description of 
demographic information for male 
and female game players. 

Five factors were found to describe 
motivation for game play: 
relationship, manipulation, 
immersion, escapism, achievement. 
Females scored higher on 
relationship, immersion and 
escapism, while males scored 
higher on achievement and 
manipulation. 
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Abstract 

The research is intended to discover the effect of instructional strategies and math anxiety on learning outcomes in 
mathematics. Factorial design 2x2 was employed in order to answer the research question how the effect of the 
instructional strategies and math anxiety on learning outcomes in mathematics. It implies that there is no single 
instructional strategy that gives better residt on learning outcomes in mathematics for all students with the different 
math anxieties. Based on this research findings mathematics vocational teachers in the field of technology and 
industry should apply several instructional strategies to serve students with different math anxieties. 

Introduction 

Learning strategies that have been developed by a math teacher at SMK result in a contribution to the 
extent of checking students' reception and interpretation of mathematical concepts and procedures, causing anxiety 
on students, especially every time they do the tasks as well as will take the test. It appears that the role of learning 
strategies in mathematics learning activities is importan. PBL learning strategy offers the freedom of students in the 
learning process. The problem in this study were: 

• Are there differences in learning outcomes between students who take mathematics learning strategy PBL with 
students who take Expository learning strategy?, 

• Are there differences in mathematics learning outcomes of students who have high levels of math anxiety 
among the strategies followed by the PBL learning and teaching strategies that follow Expository? 

• Are there differences in mathematics learning outcomes of students who have low levels of math anxiety among 
which followed the PBL and learning strategy that follows Expository learning strategy? 

• Is there an interaction effect between learning strategy and the level of mathematics anxiety on math learning 
outcomes? 


Methods 

The research was conducted at the Central Jakarta SMK 39 semester 2008-2009 school year. The method 
used is an experimental method with a 2x2 factorial design. The dependent variable in this study is a result of 
students' mathematics learning. The independent variable is the treatment of learning strategies, which can be 
divided into two groups, namely PBL as a learning strategy and the experimental group and Expository teaching 
strategy as a control group. Intervention in the form of the independent variable is the variable attribute mathematics 
anxiety. 

Data collection used a test developed by math anxiety Gilani (2002: 159) which has been tested again and the test 
results to learn metematika. For hypothesis testing is done using Analysis of Variance technique (ANOVA) 
followed by a two-lane Scheffe test '. 


Results 

Based on the analysis of data by ANOVA two lines of evidence point hypothesis in this study can be 
explained in the following description. First, there are differences in the overall learning outcomes of students who 
take math learning strategies with PBL higher mathematics achievement than students who take Expository learning 
strategies. From the calculations, the average score for a group of students who take the PBL learning strategies of 
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24. 12, while the group of students who take the Expository learning strategies have an average score of 21.90. 
ANOVA calculation results indicate that the two pathways value Fh= 40.67 which was greater than the value of Ft = 
3.91 for significance level a = 0.05 or (Fh > Ft). This means that Flo rejected and Hi accepted, so there is a 
significant difference between the application of learning strategies with PBL with Expository learning strategies on 
learning outcomes of mathematics. Scheffe test results' obtained at Ft = 41.07 and Fh = 2.70 at the significance level 
a = 0.05. Turns Fh > Ft so that Flo rejected and Fix accepted. It can be concluded that, overall learning outcomes of 
students who take math learning strategies with PBL higher than students who take the expository teaching 
strategies. 

Second, the results of the group of students learning mathematics with high math anxiety and learning 
strategies followed by PBL has an average score of 25.68, while a group of students who have high math anxiety 
and follow Expository learning strategies have an average score of 18.38. Scheffe test results' obtained at Fh = 
197.37, while for a = 0.05 significance level of 2.80 and a significance level of 0.01 to 4.22 magnitude. It turned out 
better value greater than the significance level of 0.05 and a significance level of 0.01, so H 0 rejected and Hi 
accepted. Means for groups of students who have high math anxiety, students who take learning strategies with 
learning outcomes PBL have higher math than students who take Expository learning strategies. 

Third, mathematics learning outcomes for students who have low levels of math anxiety yangmengikuti 
PBL learning strategies with lower than students who take Expository learning strategies. From the calculations, the 
average score of the group of students who have low math anxiety and learning strategies followed by PBL has 
amounted to 21.72, while the group of students who have math anxiety low and follow the expository instructional 
strategies have an average score of 23.89. Based on Scheffe test 'was obtained for Fh = 18.84, while for the 0.05 
significance level of 2.80 and a significance level of 0.01 to Ft = 4.22 magnitude. It turned out better value smaller 
than the significance level of 0.05 and a significance level of 0.01, so Flo accepted and Fix rejected. Means for 
groups of students who have low math anxiety, students who take learning strategies with learning outcomes PBL 
had lower math than students who take Expository learning strategies 

Fourth, there is an interaction effect between strategoi learning and math anxiety on math learning 
outcomes. The test results show that the second hypothesis, the results of students' mathematics learning with high 
math anxiety and learning strategies followed by PBL higher than students who had high math anxiety and learning 
strategies Expository follow. The third hypothesis test results showed that students' mathematics learning outcomes 
that have low math anxiety and learning strategies followed by PBL lower than students who had low math anxiety 
and learning strategies Expository follow. 

Results of the second and third hypothesis test indicates the interaction between learning strategy math 
anxiety in their influence on mathematics learning outcomes. ANOVA calculation results confirm these indications, 
because of the calculation obtained Fh= 176.99 which was higher than the value of Ft = 3.91 for significance level a 
= 0.05 and also larger than the significance level Ft = 6.90 for a = 0.01 or Fh > Ft. This means that Flo rejected and 
Fix accepted, so it can be concluded that, there is a highly significant interaction effect between learning strategies 
and mathematics anxiety in their influence on mathematics learning outcomes. Interaction between learning 
strategies and mathematics anxiety in their influence on mathematics learning outcomes are graphically shown in 
Figure 1. below this. 


Y 



MATH ANXIETY 
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Figure 1. Interaction between Learning Strategies and Mathematics Anxiety Impact 
on Learning Outcomes in Mathematics 

Findings 

Testing the four hypotheses proposed in this study has yielded details of the test results the following 
hypothesis. First, the results of the first hypothesis test has been successfully reject the null hypothesis that states 
there is no difference in mathematics achievement between groups of students who take lessons with and PBL 
learning strategies with groups of students who take Expository learning strategies. So overall there are differences 
in mathematics achievement between students who take the PBL learning strategies with students who take 
Expository learning strategy, ie learning outcomes of students who take math learning strategies with PBL higher 
mathematics achievement than students who take Expositoiy learning strategies. Second , the results of the second 
hypothesis test successfully reject the null hypothesis that states there is no difference in students' mathematics 
learning outcomes that have high math anxiety, among the group of students who take learning strategies with PBL 
and the group of students who take Expository learning strategies. So that there are differences in mathematics 
learning outcomes of students who have high levels of math anxiety among the following learning strategy with the 
following PBL Expository learning strategy, ie learning outcomes of students who take math learning strategies with 
PBL higher learning outcomes than students who take math learning strategies Expository. Third, the results of the 
third hypothesis test successfully reject the null hypothesis that states there is no difference in students' mathematics 
learning outcomes that have low math anxiety, among the group of students who take learning strategies with PBL 
and the group of students who take Expository learning strategies. So that there are differences in mathematics 
learning outcomes of students who have low levels of math anxiety among students who take learning strategy with 
the following PBL Expository learning strategy, ie learning outcomes of students who take math learning strategies 
with PBL inferior learning outcomes of students who take math learning strategies expository. Fourth, the results of 
the fourth hypothesis testing has been successfully reject the null hypothesis that states there is no interaction 
between PBL and learning strategies with math anxiety in their influence on mathematics learning outcomes. Thus 
the fourth hypothesis test showed that there was an interaction between learning strategies and mathematics anxiety 
in their influence on mathematics learning outcomes. 

Limitations 

First, SMK 39 Jakarta Students come from different economic backgrounds, family backgrounds, and 
neighborhoods. The similarity of the characteristics of students still require further study. Second, control of 
mathematical skills for research subjects only include the variable learning strategies and mathematics anxiety. The 
results can be influenced by other variables outside the predefined variables in this study. Other variables, such as: 
intelligence, interest, motivation, learning styles, learning environment, reasoning ability and others not controlled. 
Third, this study also does not take into account the influence of early mathematical skills of students who became 
the object of research, so it can not be known a gain score. Fourth, the mathematical material used in this study only 
mathematics X class vocational courses electronics and technology industries, namely: matrices, linear 
programming, and mathematical models fit the book Education Unit Level Curriculum (SBC) SMK 39 Jakarta in 
2007. 


Recommendations 


1. Suggestions To Teachers 

With the results of these studies as described earlier discussions, the SMK and the technology industry in 
particular, it is recommended that: 

a. Implementing learning using learning strategies with PBL as one of the alternatives in the learning of 
mathematics. With learning strategy and PBL learning achievement of mathematics as a whole proved to be more 
successful than Expository learning strategies. Learning strategies with more PBL allows terbelajarkannya 
mathematics as a whole. In other words, students are actively involved in the learning process, practice solving 
problems through discussions and access information from a variety of learning resources. 

b. In designing and developing mathematical learning programs, teachers need to pay attention to the 
characteristics of students who received math learning program. To students who have a high math anxiety would be 
more appropriate if the following learning strategies with PBL, whereas for students who have low math anxiety is 
more apt to follow the Expository learning strategies. To that end, teachers are expected to be more careful in 
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identifying the students' characteristics, the characteristics of the subject matter, and competencies to be achieved, in 
order to select and implement learning strategies more appropriate mathematics. 

c. Teachers, should strive to improve its ability to use learning strategies with PBL. This is because the 
PBL learning strategy is relatively new and unfamiliar teachers use it. 

d. Do the math student learning outcomes assessment and continuous continuously. Therefore, assessment 
tools can be developed in the form of a test or non-test with a variety of in accordance with the characteristics of the 
object being evaluated and the objectives to be achieved. 

2. To the suggestion Principal 

A process of innovation in the field of education, requiring innovation support system where it will be 
applied. To support the process of change in the paradigm of teaching in schools in accordance with the demands of 
science and technology development, the head of technology and industrial vocational schools are expected to 
provide opportunities for teachers to undertake various reform efforts in learning mathematics, especially in 
adopting strategies for learning with PBL as an alternative learning strategy mathematics in school. To implement 
this decision, in addition to providing the opportunity for teachers, also need support from the principal and school 
superintendent. Renewal process that occurs at the level of the class, will depend on the opportunities provided to 
teachers to implement the learning process seriously, as a framework of overall achievement of learning objectives. 

3. Advice To Other Researchers 

Suggested to other researchers to develop further research related to mathematics learning strategy that is 
more likely to improve students' mathematics learning outcomes, and assess other learning strategies as well as 
taking into account the characteristics of student learning outcomes related to mathematics. In connection with that, 
it is necessary to study the following matters: a) conduct another investigation by looking at the mathematics 
learning outcomes not only in terms of content, but also in terms of process and mathematics attitude, b) control the 
independent variables beyond the variables examined more closely, such as: initial test scores, student interest and 
motivation, learning styles, ways of thinking, talent and intelligence, c) conducted a study on the re-location or a 
different school levels, to determine whether the results of research conducted by the same research this, d) 
custodian chose mathematics teacher with sex or different levels of education. 
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Abstract 

This mixed methods study investigated factors that influence pre-service teachers’ intentions to use Web 2.0 
technologies in their future classrooms during teacher education course and their ability to carry out their intentions 
into actual behavior during student teaching experience. Findings revealed that pre-service teachers’ attitude and 
their perceived usefulness of Web 2.0 technologies are strong indicators of their intention to use Web 2.0 tools 
during teacher education course. One year later, participants indicated that they were able to transfer their intentions 
during student teaching experience. Use of Web 2.0 during student teaching was influenced by their perceived 
usefulness of Web 2.0 technologies, technology support, self-efficacy, and knowledge of various Web 2.0 tools. 
These results suggest that given the presence of facilitative factors, preservice teachers are able to transfer their 
intentions to use Web 2.0 technologies into actions to support student learning in their classrooms. 

Introduction 

While the growing role of Web 2.0 technologies (wikis, blogs, social networking, etc.) in education has 
brought unprecedented opportunities for both students and faculty by providing access to information and 
facilitating easy communication, at the same time this has led to the need to better prepare pre and in-service 
teachers to create socially active learning environments that encourage interaction and collaborative learning 
(Coutinho, 2008). In recognition of this need, preservice teachers must be prepared to implement advanced 
technologies in order to meet the needs of 2 1 st century learners (American Association of Colleges for Teacher 
Education (AACTE) & P21, 2010; International Society for Technology in Education (ISTE), 2008; U.S. 
Department of Education, 2010). The National Educational Technology Standards for teachers (NETS-T) 
emphasizes the necessity for pre-service teachers to gain the fundamental knowledge, skills and attitudes needed to 
incorporate contemporary tools and resources into the learning process (ISTE, 2008). 

Preservice teachers’ positive intentions toward using technologies have been shown to be a major predictor 
of their future use and successful integration in their classrooms (Myers & Halpin, 2002; Yushau, 2006). For this 
reason, a number of studies have explored influential factors that explain preservice teachers’ intentions to use 
various technologies including computers (Teo, 2009), software (Anderson & Maninger, 2007), and information 
technologies (Birch & Irvine, 2009). Although these studies have explored factors influencing the technology 
integration efforts of preservice teachers in general, little research has been conducted that examines the potential 
factors that determine preservice teachers’ intentions to use Web 2.0 technologies in a classroom. Furthermore, only 
limited studies have followed preservice teachers during their student teaching experiences to examine whether their 
stated intentions, during their teacher education programs, actually materialized into action. Cullen and Greene 
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(2011) highlighted the need to examine how intention translates into practice when preservice teachers are given a 
choice to use technology during student teaching. A combination of factors that influence both intentions and actions 
of preservice teachers would provide more robust information to guide teacher education programs, charged with 
preparing preservice teachers to use Web 2.0 technologies in their future classrooms. 

Purpose of Study 

The purpose of this study was two-fold. The first was to investigate factors that predict preservice teachers’ 
intentions to use Web 2.0 technologies in their future classrooms to improve student learning. The second was to 
explore the transfer of intentions into actions during student teaching and the factors that influenced actual use. The 
following questions guided the study: 

1. What factors best predict preservice teachers’ intentions to use Web 2.0 technologies in their future 
classrooms? How do preservice teachers describe the factors that predict their intentions to use Web 
2.0 technologies in their future classrooms? 

2. What factors facilitated or inhabited preservice teachers’ use of Web 2.0 technologies during their 
student teaching? How do preservice teachers describe the factors that facilitated or inhabited their use 
of Web 2.0 technologies in their actual classrooms? 

3. To what extent are preservice teachers able to translate their intentions for using Web 2.0 technologies 
into action during student teaching? 

Theoretical Framework 

This study used the decomposed theory of planned behavior (DTPB) as its theoretical framework (Taylor & 
Todd, 1995). The DTPB extends Ajzen’s (1991) theory of planned behavior (TPB), which focuses on the 
formulation of an intention to behave in a particular way. TPB suggests that a combination of behavioral intention 
and perceived behavioral control determines one’s actions. The DTPB explores subjective norms and perceived 
behavioral control more completely by decomposing attitude, subjective norms, and perceived behavioral control 
into lower-level belief constructs and states that behavioral intention determines behavior and that attitude, 
subjective norms, and perceived behavioral control are direct determinants of behavioral intention. The DTPB 
provides a comprehensive way to understand how an individual’s attitude, subjective norms and perceived 
behavioral control can influence his or her intention to use Web 2.0 (Ajjan & Hartshone, 2008). Moreover, it helps 
examine the relationship of factors that impact the adoption and use of new technologies more specifically (Taylor 
& Todd, 1995). 


Methods 

A sequential explanatory mixed methods design in two different phases (see Figure 1) was used. In both 
phases, quantitative data were collected from online survey and qualitative data were collected from one open-ended 
survey questions and semi-structured interviews to validate and expand quantitative results with qualitative data 
(Creswell & Clark, 2007). The results of the two phases were integrated during the interpretation of the outcomes of 
the entire study. 
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Figure 1. Visual Model for Mixed Methods Sequential Explanatory Design Procedures 


Participants 

The study took place at a Midwestern university in during Fall 2010 and then follow-up during Spring 
2012. In phase one, 189 pre-service teachers completed the online survey and 12 were purposefully selected to 
participate in the semi-structured interviews. Criteria for selection included varied majors, gender, and grade level 
interests. In the follow-up second phase, of the initial sample («=189) only 22 pre-service teachers completed their 
student teaching. Out of 22 pre-service teachers, a total of 14 completed the surveys and six were selected to 
participate in the follow-up telephone interviews based on the survey responses. Criteria for selection included 
participants who reported using Web 2.0 tools and who did not use Web 2.0 tools during student teaching. 


Data Collection 


Surveys and interviews were employed to answer the research questions for both phases of the study (see 

Table 1). 


Table 1. 

Ovei~view of Research Questions, Data Sources, & Analysis Procedures 
Phases Research Questions Data Sources 

Phase 1 Ql. Factors best predict pre- - Post project survey: 

service teachers’ intentions. DTPB scale 

- Description of predicted intent - Semi structured Interviews 
factors - Open ended survey responses 


Analysis Procedure 
QUANT + QUAL: 

- Path Analysis 

- Constant comparative 
method 
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Post project survey: 
Attitude scale data 


QUANT: 

Frequencies and percentages 


Phase 2 Q3. Factors that influence pre- Follow up data 

service teachers’ use of - online survey 


quant: 

-Descriptive statistics 


Web 2.0Q2. Transfer of 
intention into actions. 


- Semi structured Interviews QUAL 

- Open ended survey responses Constant comparative 

method 


Survey Instrument 

Phase One. The survey instrument consisted of three sections and was partially adapted from previous 
studies (Hartshorne & Ajjan, 2009). The first section included four statements about participants’ intentions to use 
Web 2.0 tools and perceptions of pedagogical advantages. The second section of the survey consisted of modified 
items of the DTPB scale with a series of 7-point Likert-scale (Strongly agree to strongly disagree) to examine 
factors that influence pre-service teachers intentions to use Web 2.0 technologies in their future classroom. Items 
focused on behavioral intention, perceived behavioral control, attitude, and subjective norms. The third section of 
the survey included six multiple choice items to determine the demographics of the participants. Demographic and 
attitude scale data were analyzed with descriptive statistics and the DTPB results were analyzed using path analysis 
to test the research hypothesis related to determining factors and to estimate the degree of the linkage between 
variables that determine intention to adopt Web 2.0 technologies (Hartshorne & Ajjan, 2009). 

Phase Two. The survey instrument consisted of two sections. The first section of the survey contained a 
question asking pre-service teachers whether or not they had used Web 2.0 technologies in their classroom followed 
by four open-ended questions exploring the factors that influenced their decisions to use or not use Web 2.0 
technologies. The second section included six multiple choice items to determine the demographics of participants. 
Frequencies of responses were tabulated and percentages were determined in order to determine if the pre-service 
teachers were able to translate their intentions of using Web 2.0 into actual actions during student teaching 
experience. 

Interviews 

The interview questions in phase one were developed based on the Web 2.0 attitude and the DTPB 
constructs to further explore teachers’ survey results and gain additional insights. Follow-up semi-structured 
telephone interviews were administered in phase two were developed based on the research questions. Qualitative 
data in both phases were content analyzed using Miles and Huberman’s (1994) constant-comparison approach. Once 
all of the transcripts were coded, each category was then re-analyzed to determine the relationships between the 
codes. The statements were examined and grouped according to the research questions and by identifying emerging 
themes that provide further explanations. 


Results 

RQ-1. Factors that best predict preservice teachers’ intentions to use Web 2.0 technologies 

The behavioral intention was a strong determinant of actual behavior or usage of Web 2.0. Regression 
results confirmed each of the three factors — attitude, subjective norm, and perceived behavioral control — explained 
a significant variance (70.8%) in behavioral intention (adjusted R2). 

Attitude. The results indicated that the pre-service teachers’ attitudes and their perceptions of the usefulness 
of Web 2.0 tools are the strongest determinants of their intentions to use Web 2.0 technologies. Regression results 
confirmed each of the three factors, perceived usefulness, perceived ease of use, and perceived compatibility, and 
explained a significant variance (68.6%) in attitude (adjusted R2). Perceived usefulness was the most significant 
predictors of their intentions to use Web 2.0 in their future classrooms. All («=12) interview participants mentioned 
that they will use Web 2.0 in their classrooms due to its potential positive impact on student learning. 
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Figure 2. Path analysis of factors that predict pre-service teachers’ intentions to use Web 2.0 technologies in the 
classroom 

Subjective norm. Regression results confirmed each of the three factors — superior, student, and peer, — 
explained a significant variance (79.5%) in the subjective norm (adjusted R2). Student influence had the strongest 
influence on the subjective norm, which, in turn, had a strong influence on behavioral intention. From the interviews 
“students influence” also emerged as the biggest factor in determining whether or not the pre-service teachers intend 
to integrate Web 2.0 into their teaching. Most («=10) of the pre-service teachers thought their students would 
influence their use of Web 2.0. For instance one participant stated, “If students want more integration of Web 2.0, 1 
will definitely use more applications to keep them engaged.” 

Perceived behavioral control. Regression results confirmed each of the three factors — facilitating resources 
conditions, facilitating technology conditions and self-efficacy — explained a significant variance (32.2%) in 
perceived behavioral control (adjusted R2). All three factors were found to influence the perceptions of behavioral 
control, which also had an influence on behavioral intention of pre-service teachers to use Web 2.0 technologies, 
with self-efficacy having the strongest influence. The interviews revealed that pre-service teachers were motivated 
to use Web 2.0 technologies due to their high self-efficacy in using these technologies. For example, one participant 
reported, “I have a lot of experience and skills where I can go online, research things and find the right Web 2.0 
tools to use.” 

RQ-2. Factors that Facilitate or Inhabit Preservice Teachers’ Actual Use of Web 2.0 Technologies 

Interview data revealed perceived usefulness of Web 2.0 tools to enhance student learning through 
interaction, self efficacy and ease of using Web 2.0 tools, availability of technology/resources, technology access 
and support in the classroom, and knowledge of various Web 2.0 tools were the most important factors that 
influenced pre-service teachers’ use of Web 2.0 in the classroom. For instance, one participant stated, “Comfort with 
technology and access to websites, Internet, and computers were main factors for my use of Web 2.0.” 

Although most of the preservice teachers used Web 2.0 technologies, a few were not able to use them. 
During the post-student teaching interview, two participants explained that even though they intended to use Web 


192 


2.0 technologies in their classrooms, they were unable to because of limited access to technology resources (e.g., 
computer labs, the Internet, blocked websites, etc.) and unsupportive cooperating teachers who were not accepting 
of new technologies. 

RQ-3. Transfer of Intentions to Actions 

A Pearson correlation coefficient was calculated for the relationship between post-course intentions to use 
Web 2.0 tools and post-teaching self-reported behavior. A strong positive correlation was found (r (14) = .531 ,p 
< .05), indicating a significant positive relationship between preservice teachers intentions and subsequent 
behaviors. Most (12 of 14) of the preservice teachers’ self-reported behaviors were consistent with their intentions 
(see table 2). According to the post-course and post-student teaching results of the 14 preservice teachers, the 
majority («=1 1) of the preservice teachers who intended to use Web 2.0 tools («=1 3), reported using Web 2.0 
technologies and only one did not intend and use these tools during her student teaching. Of those (n- 1 3) who 
intended to use these tools, two did not use Web 2.0 tools within the classroom during their student teaching 
experience. 

Table 2 

Preservice teachers ’ intentions vs. actual use of Web 2.0 tools 

Behavior 


G 

O 



Yes No 

Yes 

No 

79% 21% n =14 



Discussion and Conclusion 

This study revealed that pre-service teachers had both positive attitudes and high intentions to adopt Web 
2.0 technologies in their future classrooms to improve their student learning. The results of this study provide 
evidence that pre-service teachers’ attitudes and perceptions of usefulness of Web 2.0 tools are the strongest 
indicators of their intentions to use Web 2.0 tools. This perception of usefulness could be due to preservice teachers’ 
exposure to Web 2.0 technologies during the Web 2.0 project that helped them understand the value of using these 
technologies in the classroom. Thus, Teacher training experiences could include opportunities for pre-service 
teachers to develop actual lesson plans that integrate Web 2.0 technologies, micro-teach those lessons in teacher 
education courses, and reflect on their experiences. This might help improve their attitudes towards as well as 
enhance their perceptions of the usefulness of Web 2.0 technologies. 

The findings revealed that when preservice teachers perceive the usefulness of Web 2.0 technologies, have 
support from mentor teachers and their students, and have high self efficacy as well as easy access to Web 2.0 tools, 
they are most likely to translate their intentions into actions. This finding corroborates Bullock’s (2004) conclusion 
from his study that a combination of “effective cooperating teacher mentoring and modeling, clear expectations, 
easy access to technology and technology support, and positive experiences with technology in the classroom” can 
facilitate preservice teachers’ uses of technology in a classroom (p. 236). Therefore, this finding suggests a 
connection between intentions and actions given the presence of facilitative conditions. On the other hand, two 
preservice teachers who had intended but did not use Web 2.0 technologies reported that poor access to technology 
and a lack of support from their supervising teachers acted as inhibitors. Thus, these results suggest that cooperating 
teachers and access to technology can act both as facilitating or inhibiting factors for preservice teachers’ transfer of 
intentions into actions. To help preservice teachers successfully translate their positive intentions into actions, 
teacher educators can prepare them to take advantage of the facilitating factors and deal with the possible challenges 
of Web 2.0 technology use within a classroom. 

The follow-up results revealed that most of the pre-service teachers were able to translate their intentions 
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into actual use of Web 2.0 during student teaching. The findings support the conclusion that pre-service teachers’ 
positive intentions toward using technology are a major predictor for future use and successful integration of 
technology in their classrooms (Myers & Halpin, 2002; Yushau, 2006). Teacher educators need to look into 
promoting the translation of positive intentions towards technology integration into actual practice. Therefore, 
providing pre-service teachers opportunities to utilize their own technology enhanced lesson plans in an actual 
classroom situation with real learners might be a step in the progression between positive attitude and intentions to 
actual actions. Therefore, this study confirms that the DTPB is capable of predicting preservice teachers’ intentions 
and consequently actual behavior. Given the presence of facilitative conditions, these results substantiate a link 
between intentions and behavior. 

This study contributes to the limited body of knowledge related to the factors that influence pre-service 
teachers’ intentions and the transfer of these intentions into student teaching experience. Such finding will help 
teacher educators gain a broader understanding of the factors and craft courses, programs, and experiences that take 
these factors into consideration, and better prepare teacher to teach with Web 2.0 technologies. 
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Abstract 

Mobile devices such as the iPad have been dubbed a potential "miracle" device for special and general education; 
however, evidence-based implementation strategies have yet to emerge. Project MODS (Mobile Online Devices for 
Success in Education) is exploring the use of iPads in a K-6 merged general and special education teacher 
preparation curriculum, generating frameworks for implementation, and evaluating these frameworks for feasibility, 
social validity, and effectiveness. Using a design-based research approach, we are simultaneously developing design 
principles which inform our theory and practical heuristics that guide the instructional intervention. This paper 
describes the design and formative evaluation of a mass deployment of 40 iPad mini mobile devices in a merged 
teacher preparation program at the University of Hawaii at Manoa. The phase of the project described here focused 
on developing technology infrastructure to support instructors and students in the integration of the iPad into the 
teacher preparation program. Using an iterative development process, we have designed and implemented a solution 
for managing and administering the devices and supporting instructors and students. Results from our first round of 
formative evaluation indicate that while our support infrastructure is working well from the perspective of 
technology management, more attention needs to be given to supporting instructors and students in the use of the 
iPad minis for teaching and learning. We believe this descriptive case report will be of value to instructors, 
technology coordinators, and other stakeholders who are considering a mass mobile device implementation. 

Introduction 

The purpose of this descriptive case report is to share our process of planning and formatively evaluating a 
mass deployment of iPad Mini devices to 35 teacher candidates and 16 faculty members in a dual licensure teacher 
preparation program at the University of Hawaii at Manoa. This report is timely, as more and more educational 
institutions are looking at mobile devices, particularly tablets, as useful and promising educational technology. 
However, mass deployments of tablets can be challenging. Tablets are typically personal devices that are intended to 
be used by a single person, yet mass deployments of tablets, such as in a mobile lab scenario, often require that the 
devices be passed from person to person and that they be quickly reconfigured for different usage scenarios, 
different classes, and different students. Implementation scenarios can vary broadly, given that different educational 
institutions will naturally have different needs. It is therefore critical for institutions to identify their specific needs 
and determine how their needs can be met within the constraints of the existing support structures for mass mobile 
device deployment. We explain how we did this in the current report, along with the findings from our first round of 
formative evaluation. We believe this descriptive case report will be of value to instructors, technology coordinators, 
and other stakeholders who are currently considering or are in the process of rolling out a mass mobile device 
implementation. While our case focuses specifically on iPad Minis, many of the lessons we report could be 
generalized to any tablet implementation scenario. 

The University of Hawaii at Manoa is developing an innovative merged elementary and special education 
teacher preparation program. This program merges elementary-level (k-6) special education (SPED) and general 
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education teacher training into a seamless curriculum. Beginning Fall 2013, 35 beginning teacher candidates 
enrolled in this two-year, field based program in which all classes are co-taught by special education and general 
education faculty. This B.Ed. dual licensure teacher preparation program is innovative for our College of Education, 
promoting culturally responsive pedagogy and practice, co-teaching, universal design for learning (UDL), response 
to intervention (Rtl), data-based decision-making, and instructional technology for supporting inclusive education. If 
successful, this dual licensure model could lead to universal adoption throughout the college. We maintain that 
technology integration is essential for programmatic success. 

Project MODS (Mobile Online Devices for Success in Education) is the technology integration “arm” of 
the general and special education dual licensure curriculum. It is a joint effort of the Department of Special 
Education and the Department of Educational Technology to explore how to use mobile devices to integrate 
instructional and assistive technology and universal design for learning concepts into the experimental merged 
teacher preparation program. To date, 35 teacher candidates have been equipped with iPad Minis and apps consistent 
with curriculum goals. These iPad Minis will be used by the teacher candidates for the duration of the curriculum. 
The enhancements Project MODS is working towards by using these iPad Minis relate to infusing educational 
technologies and assistive technologies throughout the course sequence and spectrum of learning activities for 
teacher candidates who are enrolled in the program. The goals of Project MODS include 1) to gain operating 
experience and develop expertise among contributing faculty members and students in effectively integrating mobile 
devices into teacher preparation, 2) to establish, refine, and disseminate practice-based heuristics and best practices 
regarding the use of mobile devices for teacher training, and 3) to share what we have learned about implementing 
mobile devices for teacher training in practice. 

By generating and sharing practical and theoretical knowledge from our project, we endeavor to contribute 
to the small but growing base of literature that explores how mobile devices like the iPad Mini can impact 
education. In the field of education, mobile devices, particularly Apple's iPad tablet, are perceived by many to be a 
potential "miracle" device (Brandon, 2011). Although the popular press espouses the promise of mobile devices for 
education, there is a paucity of empirical research that supports the impact of these devices on learning outcomes. 
Similarly, there is little in the way of best practices for implementation of mobile devices for learning. As with any 
technology-related instructional intervention, it must not only be well planned, but also “most skillfully 
implemented and managed, if the theoretical benefits are to be reaped in practice” (Romiszowski, 2004, p. 16). 
Without a sufficient empirical base of research to support the underlying assumption that use of mobile devices 
indeed positively impacts learning, we must remain at best cautiously optimistic that mobile devices are an effective 
learning tool and at worst skeptical that this underlying assumption is ultimately specious. Further, without practical, 
experientially-derived heuristics for implementation in real-world settings, the design of interventions that have the 
potential for meaningful impact must be inferred, derived, guessed, and in some cases contrived. Regardless of the 
paucity of empirical evidence and practical experience guiding the implementation of mobile devices in education, 
we would be remiss to ignore tools that hold the promise of ubiquitous access to information and are able to be 
individualized to specific learner characteristics. 

Given the explosion of mobile device usage in education over the past decade, the decision to focus our 
attention on tablets seems rather obvious. Indeed, in Flwang and Tsai's (2011) review of research trends from articles 
published between 2001 and 2010 in six major educational technology journals (BJET, C&E, ETS, ETR&D, JCAL, 
and IETI), significant increases were found in the amount of articles being published on mobile and ubiquitous 
learning research. In fact, over twice the number of articles (103 vs. 51) were published in the last three years of the 
authors' review (2008-2010) than in the seven years before that (2001-2007). A push to establish a “one-to-one” ratio 
of devices to students through tablet initiatives is evident nationwide. Recently, the Los Angeles Unified School 
District announced plans to distribute 640,000 iPads to students by the end of 2014 at an expenditure of one billion 
dollars (Blume, 2013). Last year, the San Diego Unified School District rolled out 27,500 iPads to students at a 15 
million dollar expenditure (Kucher, 2012). Most recently, the New York Times reported that every student and 
teacher in 24 middle schools of Guilford County, North Carolina received an Amplify tablet — 15,450 in all (Rotella, 
2013). The jury is no longer out. Mobile devices have caught the attention of educators, administrators, and policy 
makers. 

In Flawaii, governor Neil Abercrombie and the Flawaii Department of Education are promoting a one-to- 
one laptop/tablet program in which each student in Flawaii public schools will be provided with a computing device 
(Vorsino, 2012). Our dual licensure students will need to be highly proficient in the implementation and integration 
of computer technologies in their curricula and classes, as it will soon be the case that each of their students will 
have their own tablet or laptop. As instructional technologists, we recognize this as a tremendous opportunity to 
impact student learning. Flowever, meaningfully and ethically integrating technology is fraught with challenges, and, 
generally speaking, mass implementations of technology to enhance student learning has a hit-and-miss track record. 
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Clearly there is a need to equip our pre-service teachers with the knowledge and skills to effectively use technology 
as a tool to promote learning and instruction, yet we also recognize the need to employ strategies that will result in 
pre-service teachers developing positive perceptions about using technology. Promoting positive technology 
perceptions will increase the likelihood that pre-service teachers will accept and use the technology tools they are 
provided (Cheon, Lee, Crooks, & Song, 2012). Part of a robust strategy for impacting technology perceptions and 
usage is training pre-service teachers to use the technology tools that they will most likely be using when they 
transition to in-service teaching. 

Mobile devices as learning tools: Benefits and drawbacks. 

The a huge amount of hype with little evidence of educational impact related to these devices raises 
concerns that technology is being pushed for technology's sake. However, from a less skeptical perspective, these 
mass deployments of technology for schools present tremendous opportunity for technologists to enact positive 
educational change. Indeed, mobile devices hold much promise for education. From the perspective of ultra- 
mobility, long battery life, and relative affordability, mobile computing devices are attractive as learning tools 
because they make it possible to provide each student with computing devices that can be used all day and easily 
transferred from class to class. In addition to these physical characteristics, educators and researchers see mobile 
devices as having a number of capabilities that are particularly desirable for enhancing learning. For example, 
Mueller and colleagues (2012) found that Blackberry devices provided an effective means for university students to 
communicate and organize tasks and group work, and allowed students to engage in a social support system. 
Enriquez (2010) argued that, for students at a community college, mobile devices provided for better teacher-student 
interaction, immediate assessment and feedback capabilities that allowed the instructor to be more responsive to 
student needs, and better overall student engagement. In their study on university students using iPads at Purdue, 
Rossing and colleagues (2012) discovered that students had positive perceptions about how these mobile devices 
afforded access and availability to information, facilitated sharing and collaborating with others, were adaptable to 
different learning styles, and were convenient and easy to use. Mobile devices provide just-in-time and on-demand 
access to vast amounts of information. Given this and the potential for mobile devices to positively impact students 
and instructors' communication, organization, and social interaction, it is hardly surprising that these devices have 
become so popular. 

The ever-increasing popularity and capabilities of mobile devices have inspired a growing number of 
research studies looking specifically at using mobile devices to enhance student learning. For example, in a study on 
a one-to-one initiative in Singapore in which 39 ninth graders used an iPod Touch for one semester, Koh and 
colleagues (2011) reported that students had overwhelmingly positive perceptions regarding their learning and 
engagement. In another study that looked at the learning outcomes and student satisfaction of 33 fifth graders in 
Taiwan using PDAs to learn social studies, Shih and colleagues (2010) found significant improvement in student 
achievement and high levels of satisfaction, and noted that teachers reported improvements in terms of affect and 
level of student participation. Further, researchers looking into the impact on learning and instruction of iPads in two 
upper elementary school classrooms found improvements in student independence and self-efficacy (Reid & 
Ostashewski, 2011). Hwang and Chang (2011) compared students' performance on test items using a quasi- 
experimental research design and found significant differences in achievement between the experimental group and 
control group. The experimental group was provided a specially designed mobile learning environment that ran on 
Windows Mobile PDAs, and the control group was given PDAs that were equipped with a more traditional e- 
learning environment. Differences in achievement between the groups were attributed to increased motivation and 
time-on-task using the specially designed mobile learning environment. Positive outcomes using mobile devices 
have also been realized in the field of special education. Fernandez-Lopez and colleagues (2013) conducted a pre- 
experimental study in which they investigated the impact of Picaa running on iPad and iTouch devices on 
development of learning skills for individuals with special education needs. Although limited by the pre- 
experimental nature of their research, they found that Picaa promoted learning and was suitable for supporting 
adaptations often needed by children with special needs. These are but a few examples of studies that connect the 
use of mobile devices with positive learning outcomes. However, a recurring theme in these and other studies is the 
maxim that positive impact on learning cannot be attributed to mobile devices alone, but instead is closely tied to the 
manner in which the devices are implemented, the instructional strategies used, and the alignment between the 
capabilities of the mobile devices and identified pedagogy. 

Successful implementation of mobile devices for education requires significant planning, which should 
start prior to implementation and continue throughout the implementation effort (Moran, Hawkes, & El Gayar, 
2010). Understanding the importance of planning is critical in light of many issues that have been identified 
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regarding implementation. For example, issues that can impact the usability of these devices include their relatively 
small screens, limited input options, and low computational power (Ting, 2012). Educators wishing to implement 
mobile devices need to acknowledge these limitations and design their mobile-ready instruction accordingly. 
According to Banister (2010), other potential impediments to implementation include the need to control for the 
device being used as a tool and not a distraction. The author suggests identifying ways to monitor and track the ways 
students are using the devices so as to be able to counteract any inappropriate or off-task usage. Flenderson and 
Yeow (2012) discuss issues identified when implementing 50 iPads in a school setting. One such issue was the time 
involved in installing, configuring, and maintaining a mass deployment of mobile devices. They stress the need to 
have a plan for recharging the devices, deploying applications, making backups, and maintaining the physical 
devices. They also point to the difficulty in identifying appropriate applications, given the multitude of educational 
applications available in the various app stores. These and other issues underscore the need for solid planning that 
acknowledges implementation issues and identifies creative solutions. A well developed plan will increase the 
likelihood of mobile device implementation success. 

Moran and colleagues' (2010) argue that planning in mobile device implementation is critical, and is 
especially relevant when considering supporting pre-service teachers' use of such devices. In Wishart's (2009) case 
study of in-service and pre-service teachers using Windows Mobile PDAs, she found that over half of all teachers 
had given up using the devices just five months after they were issued. There was no clear agreement as to why the 
pre-service teachers abandoned the devices, but some indicators point to usability issues discussed previously 
(screen size, limited input options) as well as issues related to losing data. Conversely, a 2007 study of 45 pre- 
service teachers who were well supported in using iPods in a Midwestern college of education reported that teachers 
gained confidence using the devices, enjoyed creating and publishing content with the devices, and recognized the 
devices' value as instructional tools (Keengwe, Pearson, & Smart, 2009). Teachers, particularly pre-service teachers, 
need to be provided adequate assistance and support for using mobile devices for learning. When support is 
appropriate and sufficient, teachers are able to recognize and leverage the educational potential of the devices. 

Methods 

Design-based Research 

In line with AECT 20 1 3's theme of “Innovate! Integrate! Communicate!” our perspective on innovation 
includes avoiding ill-conceived and poorly researched strategies and techniques that have consistently failed in the 
past. Our research is currently developing best practices for mobile device implementation, generating theoretical 
frameworks for implementation, and evaluating our work for feasibility, social validity, and impact. To this end, we 
are implementing an design-based research approach. DBR seeks to develop solutions to specific, complex, and 
important educational problems in real-world contexts while at the same time collecting internally valid data to 
advance educational theory (Reeves, 2006). It is a cyclical process that typically advances in three phases or cycles. 
To date, we have completed two cycles (exploration & analysis and design & construction), and we are in the 
process of completing the third cycle of evaluation & reflection (McKenney & Reeves, 2012). This third cycle is 
focusing on evaluating our implementation for soundness, viability, effectiveness and impact. We will pursue three 
iterations this three-cycle process as the first cohort of students progresses through the merged program (approx. 2 
years). 

For six months before the program began, we focused on development of our technology management 
infrastructure, formulating methods and processes for integrating technology into the dual licensure curriculum, and 
planning how we would evaluate our work. We set about establishing our initial solution using David Jonassen's 
model for iteratively developing solutions to design problems (Jonassen, 2011). The first step was to identify the 
requirements (functional specifications) and constraints of the local design problem, after which an iterative process 
of sketching, modeling, and prototyping of a solution was undertaken. Each successive iteration of the design was 
informed by beliefs (informed by data and experience) and identified constraints, thus tightening the scope of the 
design, and ultimately resulting in a design that was satisfactory. Requirements were identified by consulting the 
literature, reviewing guidance provided by the vendor (in this case, Apple), and by consulting with team members. 
Constraints for the project were identified similarly. Requirements, constraints, and challenges were not established 
a-priori, but emerged over the course of six months of analysis, exploration, and design. A full description of our 
design process is beyond the scope of this report; however, the reader is referred to Schmidt, Saban, and Ho (2013), 
who provide a full narrative of the design and development process. 

Currently, teacher candidates have begun their first semester in the program and are using iPad Minis in 
their learning and instruction. During this and over the next three semesters, we will formatively evaluate instructors' 
and teacher candidates' perceptions and usage of the iPad Minis twice a semester, using the formative evaluation 
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data to implement changes to our methods and processes in an effort to maximize the instructional and educational 
impact of our technology intervention. In the last semester of the program, we will conduct a semi-summative 
evaluation. While the interventions and theory we are developing will be specific to our local context, our goal of 
developing mobile technology-mediated, evidence-based practices is in line with the broader implementation 
science movement (Smith, Schmidt, Cook, & Edelen-Smith, 2013). 

Formative Evaluation 

Given the challenges of sound technology integration, we recognize the need for systematic and thoughtful 
implementation of technology in this curriculum. Hence, a formative evaluation of our technology integration effort 
was conducted in the Fall of 2013. The purpose of the formative evaluation was to explore perceptions of teacher 
candidates and instructors regarding the integration of iPad Minis into the merged elementary and special education 
program. The questions that guided our inquiry were: 

RQ1: How do teacher candidates and instructors perceive the use of iPad Minis as an educational tool in 

the merged elementary and special education program? 

RQ2: How do teacher candidates and instructors perceive the support they receive from Project MODS for 

meaningfully integrating iPad Minis into the merged elementary and special education program? 

These questions were inspired by the literature, particularly by those studies that underscore the importance of 
planning to successful mobile technology implementations (e.g., Cheon et ah, 2012; Moran et ah, 2010) and those 
studies that have identified potential “pitfalls” associated with mobile devices in education (e.g.. Banister, 2010; 
Henderson & Yeow, 2012; Ting, 2012). 

Anonymous surveys were developed separately for teacher candidates and instructors. The teacher 
candidate survey consisted of four closed-choice and nine open-ended questions that focused on perceptions of 
usefulness, quality of service and support, and educational impact of the iPad Minis. The instructor survey consisted 
of five closed-choice and five open-ended questions that focused on perceptions of quality of service and support 
and identifying areas where further support might be needed for integrating the iPad Mini meaningfully into 
curricula. The surveys were developed using Google Forms, after which they were piloted with four faculty 
members and cleared with exempt status from the campus IRB before being disseminated to teacher candidates and 
instructors. After piloting and IRB clearance, invitations were sent to teacher candidates and instructors via email. 
Invitations were sent to 35 teacher candidates and 16 instructors. These invitations were followed one week later 
with a separate email message reminding participants to fill out the survey forms. Response rate for teacher 
candidates was 91% (32/35), and for instructors was 50% (8/16). The lower response rate for instructors is attributed 
to the fact that some of the instructors have not yet begun teaching in the program, and other instructors are not 
classroom instructors, but instead are focusing on training other merged program instructors in such areas as co- 
teaching and culturally responsive teaching methods. 

Survey data were analyzed using descriptive statistics for closed-choice responses and open-coding for 
open-ended responses. Closed-choice responses were input into a spreadsheet from which descriptive statics such as 
mean, median, standard deviation, etc. were established. Open-ended responses were input into tables in a word 
processor and reviewed by the authors. Authors' review of the open-ended questions used a simple method of 
searching for trends and themes among participants' responses, then creating descriptors for those trends and themes, 
and associating representative firsthand quotes from participants with those descriptors. Following this, the authors 
got together and discussed their findings. The authors agreed that no formal inter-observer agreement was necessary, 
given that the focus of this evaluation was on improvements to processes and supports. However, the authors did 
note remarkable similarity in the trends and themes that were identified. 

Findings 

Perceptions of the iPad Mini as an Educational Tool 

The majority of questionnaire items focusing on perceptions of the iPad Mini as an educational tool for 
both the instructors and the teacher candidates were open-ended. In order to meaningfully evaluate responses, we 
were interested in how often participants used the iPad Minis, as well as their comfort level using the devices. We 
anticipated these data would help shed light on the open-ended responses. For instance, if a participant is 
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uncomfortable using the device and does not use it on a daily basis, it would not be surprising to find less positive 
perceptions of the device as an educational tool than, say, a participant who is very comfortable with the iPad Mini 
and uses it for many hours every day. Participants' reported daily usage of and comfort level with the iPad Mini are 
reported in Table 1. From these data, differences in teacher candidates and instructors usage and perceptions of 
comfort are evident, we find that, in general, teacher candidates are more comfortable with the iPad Mini and use it 
more on a day-to-day basis than instructors. In addition, nearly half of the teacher candidates are very comfortable 
using the iPad Mini, and over half use it for more than 4 hours a day. Over two-thirds of instructors, however, use 
the iPad for three hours or less a day, and half of them report being less than comfortable with using the device. 

Findings from the open-ended questions indicate that instructors view the iPad Minis as having the 
potential to engage teacher candidates and to be useful as an educational tool. They appear to be pleased with the 
accessibility to information, ability to take notes, and ability to participate in technology-mediated class activities 
that the iPad Mini provides. Flowever, they are unsure as to how to realize the potential of the devices. Of note is 
that the instructors did not mention teacher candidates using iPad Minis as tools for their own instruction with their 
own students, but instead as tools for their own individual learning. In addition, teacher candidates were asked to 
describe the way instructors were integrating iPad Minis into their instruction. Findings indicate that one instructor 
is using the iPad Mini extensively and that teacher candidates find this instructor's teaching to be helpful, useful, and 
interesting. However, we also found that many instructors reported they are not using iPads, but would like to use it 
more, or are instead using laptops. One instructor noted, “I tend to use my computer in my class as I am used to it 
and it has a full sized keyboard.” As teacher candidates report, this is problematic in that it creates uncertainty and 
confusion around how they can implement iPads in their own instruction, and it creates inconsistency in how 
technology is being used by instructors and and teacher candidates. The inconsistency is highlighted by one teacher 
candidate's comment, “The modeling could use improvement.” If instructors are using laptops, they are not 
modeling the use of the iPad Mini. However, despite these challenges, teacher candidates take a positive view of the 
iPad Mini as an educational tool in general, reporting that they find using it in the merged program to be engaging, 
useful, and innovative. 

Table 1 


Participants' reported daily usage of and comfort level with the iPad Mini 


Amount of time iPad Mini used daily 


Instructors 

Teacher Candidates 

More than 6 hrs 

0.0% 

18.8% 

4-6 hrs 

12.5% 

37.5% 

1-3 hrs 

50.0% 

21.9% 

Not used daily 

37.5% 

21.9% 

Comfort level using the iPad Mini 


Instructors 

Teacher Candidates 

Very comfortable 

12.5% 

47.0% 

Comfortable 

37.5% 

6.0% 

Somewhat comfortable 

25.0% 

16.0% 

Uncomfortable 

12.5% 

25.0% 

Very uncomfortable 

12.5% 

6.0% 


Perceptions of support for meaningfully integrating iPad Minis into the merged program 

Given that the majority of the Project MODS effort to date has been on effectively planning and 
implementing our technology infrastructure, we were keen to investigate teacher candidate and instructor 
perceptions of the support we have provided them since they started the merged program. We took a view of support 
as being focused on providing support for managing and maintaining the iPad Minis as well as support for using the 
iPad Minis as an instructional and learning tool. As to support for managing and maintaining the iPad Minis, data 
indicate overwhelmingly positive perceptions, with only one area emerging as a potential target for change. Teacher 
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candidates are not given full control over what software they are allowed to install and uninstall on their devices. 
Many students reported frustration at not having control over deleting and syncing apps on their devices. However, 
when asked what they would change about device management and maintenance, the majority of participants 
responded, “Nothing.” 

Perceptions were markedly different regarding support for using the iPad Minis as an instructional and 
learning tool (Table 2). Data indicate that, on average, training is viewed as somewhat adequate by both instructors 
and teacher candidates, with responses deviating by about 25% per group (instructors' ,vt/= 1 .56; teacher candidates' 
sd= 1.29). These findings align with our analysis of open-ended responses from participants. Instructors reported an 
overwhelming desire for more training. Many indicated that they feel unprepared to use the iPad Mini as an 
instructional and learning tool and admit to not knowing how to implement iPads in their courses. One instructor 
stated, “I want to try out more - experiment - but don’t know how.” They note that they need more time to learn 
how to use the tool, as well as more time to co-plan their teaching with the Project MODS team. Clearly, 
improvements in instructor support are needed. This also helps to explain why instructors are not using iPads as 
much as desired in their own instruction and instead using tools with which they are more familiar. Areas that 
instructors feel would be helpful to focus on for training are presented in Figure 1. 

Table 2 

Participants' reported perceptions regarding the adequacy of training for using the iPad Mini as an instructional 
and learning tool. 


Adequacy of training for using the iPad Mini 



Instructors 

Teacher Candidates 

Very adequate training 

12.5% 

28.0% 

Adequate training 

37.5% 

19.0% 

Somewhat adequate training 

12.5% 

25.0% 

Somewhat inadequate training 

25.0% 

22.0% 

Inadequate training 

12.5% 

6.0% 
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Faculty perceptions of training needs 



Figure 1. Instructors' perceived training needs. 


Discussion 

In our report, we have introduced our project to integrate mobile technology (iPad Minis) into a new 
merged teacher preparation program in which teacher candidates have the opportunity to become certified to teach in 
both general education and special education. We provide a review of the literature in which we present an overview 
of the benefits attributed to mobile devices, along with a discussion of the potential “pitfalls” associated with these 
devices. In addition, we describe our particular case of using a design-based research approach to designing, 
developing, and evaluating a solution for managing a mass deployment of iPad Minis. In this section, we discuss our 
findings and the lessons we have learned in our DBR process. 

Generally speaking, our technology infrastructure appears to be working well. However, it is clear that 
more instructor and teacher candidate support is needed. This presents both challenges and opportunities. A 
challenge is that the support and training needs to have multiple foci. From the perspective of the iPad Mini, support 
will need to be multi-tiered, focusing on general device usage, software specific training (e.g., apps), and more 
conceptual/pedagogical training on how to use the iPad Mini as an instructional and learning tool. One way we 
could do this might be to provide a few weekly scheduled “help desk” or “genius bar” times in which instructors and 
teacher candidates can work directly with a support team member. Another way might be to compile information 
and training into our existing support website. While these ideas have been met positively, we admit that they are 
insufficient for approaching the larger problem of training instructors and teacher candidates to use the iPad 
effectively and meaningfully as an instructional tool. Given that design-based research should maintain a focus on 
iterating instructional interventions in an effort to maximize educational impact (McKenney & Reeves, 2012), we 
see this as the area that should receive the most focus during our next design iteration, as it is arguably the area 
where the potential for establishing impact is greatest. The challenge is complex, however, as instructor needs differ 
from teacher candidate needs. Instructors need support for using the iPad mini for instruction, whereas teacher 


202 



candidates need support for using the iPad Mini for their own learning, as well as future instruction with actual 
students. While we do not have solutions to these issues yet, we are working towards them in our next iteration of 
DBR cycles. 

One of the goals of our DBR effort is to establish heuristics for mass implementation of mobile devices. 

The most important heuristic we have been able to derive from our processes so far is to plan for iteration. Any 
educator considering integrating mobile devices into a curriculum should allow for exploration and expect that 
initial solutions will fall short of design goals. We have also learned that any mass implementation requires 
significant planning, a heuristic that is strongly supported by the literature. Our planning to date has been primarily 
focused on the management and administration of our iPad Mini fleet, so it is hardly surprising that instructors and 
teacher candidates desire more support in their implementation of iPad Minis as instructional and learning tools. 
However, the importance of a management solution for large amounts of devices cannot be understated. As 
educators, we are often overwhelmed with responsibilities. Trying to cope with a mass deployment of mobile 
devices without a solid management plan, even a small implementation such as ours, is likely to be untenable. The 
heuristics outlined above are not specific to mobile devices. Iteration and planning are key to any successful 
technology integration. The design and evaluation effort reported here merely provides more evidence of this. 
However, mass iPad deployments have key differences when compared, for example, to mass computer or laptop 
deployments, such as being designed for ultramobility and for usage by a single user. These differences should not 
be underestimated. Supporting mass deployments of tablets takes time, effort, creativity, and research. 

We conclude on a cautionary note. While there is much hype around mobile devices, the reality is that 
iPads are not going to “save” education. Often touted at Apple-sponsored events as “magical,” there is really nothing 
special about them. They are fundamentally hardware that runs software. What makes them compelling for 
education is the combination of ultramobility, long battery life, access to information, low-cost applications, a 
multitude of applications, ease-of-use (and relative difficulty to break), along with a relatively intuitive touchscreen 
interface. However, these devices are quite limited in many ways. They have no physical keyboard and mouse, and 
are therefore not ideal content creation devices. Implementing for consumptive learning activities that require 
reading, watching videos, etc. does not require any particularly creative feats of imagination. However, for activities 
that require significant writing or media production, iPads seem more appropriate as technology that can augment, 
but not drive, content creation. And while these devices have the potential to connect students and allow for 
interesting and useful cases of networked collaboration, realizing this potential is an area that is not yet well 
understood. Understanding these limitations is key when developing pedagogical interventions that integrate mobile 
devices in general and iPads specifically. 
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Ill-Structured Problems, Scaffolding and Problem-Solving Ability of Novice 

Nursing Students 

Debra M. Stark 

Ill-structured problems (ISPs), such as those encountered in everyday life, often possess either no clear or 
multiple solutions and also present a degree of uncertainty about concepts, rules and principles that might be 
necessary for problem organization and solution. Jonassen (2011) recommends scaffolding support for learners as 
they work to solve ISPs. The present research focuses on the impact of scaffolding on ill-structured problem solving 
(using case examples) by novice nursing students. The study employed a quasi-experimental, mixed-method design 
to investigate the effect of two forms of scaffolding — question prompts and alternative perspectives — on ill- 
structured problem-solving and metacognition. Think-aloud protocols and self-reflections also were used to assess 
the effect of scaffolding techniques on problem solution. Employing question prompts was found to have an effect 
on two problem-solving constructs: problem representation and monitoring/evaluation. Using alternative 
perspectives was found to have an effect on problem representation and constructing arguments to support solutions. 

All individuals need to be competent problem solvers both on the job and in everyday life. Those working 
in health care, in particular, need to be capable in solving problems because of the dynamics of the job environments 
in which they find themselves. Delivery of good, quality healthcare is a continuing challenge due to increased and 
changing technologies in the workplace, increased patient complexity, and issues related to healthcare access 
balanced against the demand to curtail overall system costs, among other concerns (Benner, Sutphen, Leonard, & 
Day, 2010). These and other transformations of healthcare impact both nursing practice and nursing education 
(Benner et al., 2010). Because nurses need high degrees of knowledge, skills and problem-solving abilities in order 
to deliver complex therapies, address patient social issues, and navigate current healthcare settings (Benner et ah, 
2010), exposure to and practice with the cognitive processes that go into solving ill-structured problems (ISPs) is 
encouraged at the earliest possible time in nursing education. 

Achieving this early introduction can be a daunting task for both educators and learners. Due to the 
complex nature of ISPs, learners can experience high cognitive loads during the problem solving process. When 
learners begin the process of solving ISPs, they must bring into working memory all they know about the situation 
(Jonassen, 2011). Because of the limited capacity of working memory, learners can become overwhelmed with the 
amount of previous knowledge and new knowledge that has to be “juggle[d]”during the initial stages of problem- 
solving (Jonassen, 2011). Learners need support as they acquire the skills needed to solve complex ISPs. 

Scaffolding can be provided as an important instructional support to help the learner manage increased 
cognitive load demanded in ill-structured problem-solving situations. The idea of scaffolding support stems from 
Vygotsky’s (1978) Zone of Proximal Development. Based on his ideas and research, scaffolding would provide the 
level of support learners need to achieve higher levels of ability in problem-solving than would be possible without 
this (Vygotsky, 1978). 

A survey of research related to the impact of scaffolding on ill-structured problem solving reveals that 
‘question prompts’ have been used to this end (Ge & Land, 2003; Ge, Planas, & Er, 2010) because questioning can 
be fundamental in guiding human reasoning (Graesser, Baggett, & Williams, 1996). Therefore, development of 
prompts that guide the learner’s thinking while they work to understand the problem and generate solutions is 
essential (Jonassen, 2011). Questions can aid in problem-solving activities by increasing understanding of causal 
reasoning associated with the situation as well as in more fundamental skills such as planning, setting goals, and in 
justifying the problem and problem solutions (Jonassen, 2011). Within the larger study by the Carnegie Foundation 
on the Preparation of Professionals, Benner et al. (2010) investigated the state of nursing education within the 
United States. During their study, an exemplary nursing teaching strategy cited by Benner et al. (2010) was the use 
of questioning that helped students integrate skill and ‘know-how’ to support their development of ill-structured 
problem solving abilities. 

In the absence of prior experience by the problem solver, experiences of others solving the same problem 
has the potential to facilitate ill-structured problem solving by exposing the learner to alternative perspectives on the 
situation (Jonassen, 2011). The use of alternative perspectives centers on experts and/or individuals with knowledge 
in the field or situation telling their version of how they perceive the problem(s) and how they would go about 
finding a solution. Bruner (1990) showed that people organize their life and experiences around story telling as this 
means of information transmission because crafting stories it takes less cognitive effort. In solving problems within 
real world contexts people many times use stories to help explain their actions and thoughts to others (Jonassen & 
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Hernandez-Serrano, 2002; Schon, 1993). Choi and Lee (2009) found the use of alternative perspectives, told as 
stories, to supported participants as they worked through difficulties in classroom management and improved their 
abilities in problem solving. Of note, nursing faculty often use stories of clinical practice situations for teaching. 
Instructors provide detailed information on the patient condition and situation, nursing thoughts, and actions as they 
planned and executed patient care (Benner et al., 2010). Jonassen (2011) described alternative perspectives being 
used in place of prior experience when solving ill-structured problems. 

The current study investigated the effects of scaffolding using question prompts or alternative perspectives 
on the ill-structured problem-solving abilities of novice nursing students with respect to (a) problem representation, 
(b) developing solutions, (c) making justifications, and (d) monitoring and evaluation. Specifically, the research 
question posed whether any differences would emerge when comparing three groups in this type of learning 
situation: those students who received question prompts to help solving ISPs, those students who listened to 
alternative perspectives while solving ISPs, and those students who served as the control and received no scaffolding 
support for problem solving. 

Methods 

The study employed a quasi-experimental, concurrent, mixed-methods design to investigate the research 
question. The participants in the study were 68 students enrolled in three sections of a beginning nursing course 
given at a small, Midwestern, private nursing college in the United States. The participants had a wide variety of life 
experiences coming into the course. What they had in common was the fact that this course was one of their first 
introductory nursing classes. Participation in the study was voluntary. Approval was obtained from the governing 
internal review boards. 

The course introduced the student to the concepts of health promotion, access to healthcare, health 
literacy, healthcare costs and the role of the nurse. The course was taught once a week with 120-minute 
classroom sessions. The study was integrated into the course curriculum to allow assessment of the study 
participants in a natural classroom setting. The rationale for using this beginning nursing course as a setting 
for the research study stems from the idea that improving nurse’s ill-structured problem-solving and 
metacognitive skills should begin at the earliest stages of their nursing education. 

Data Collection Techniques and Procedures 

All participants read problem cases, one per week, for four weeks and completed a problem 
solution report for each one. Participants were asked to complete a self-reflection at the end of the study 
processes after the last case report had been filed. Of the 68 participants, 65 completed all four of the 
problem cases and 58 completed the self-reflection report. From the last grouping, 6 participants also 
participated in the” think-aloud protocols” for each of the four cases. 

To complete the problem solution reports the participants were asked to read the problem case and 
then do the following: (a) define the problem, (b) propose solutions, (c) provide evidence for their 
solutions, and (d) evaluate the solutions. The participants’ responses to these case reports comprised part of 
the data used to answer the research question. 

The problem cases were accessed using the learning management system (LMS) during their 
regular scheduled class time. The participants typed in their responses to the problem cases during the class 
time in a text box displayed on the screen below each of the cases. During the baseline and final problem 
case, scaffolding was not provided for any of the participants. In problem Case 1 and Case 2, the 
participants from the experimental groups received scaffolding in the form of question prompts or by 
listening to alternative perspectives via an online link. The control group did not have access to these types 
of support. 

Participants submitted their answers through the LMS once they had completed the assignment. 

After the problem solution reports were submitted, the course instructor printed a copy of each participant’s 
problem solution report, removed identifying information and then assigned it a number code. The same 
number code was used for each participant throughout the study. The course instructor then gave the 
solution reports to the researcher. 

The six participants (two from each section of the course) who participated in the think-aloud protocols 
were videoed in a separate room during each of the problem cases. The verbalizations of the participants from each 
group who participated in the think-aloud protocols were transcribed into a word document by the researcher. The 
word document was then used to analyze the thinking process of the participants as they worked through the 
problem-solving cases. 
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All participants were asked to complete a self-reflection after the final problem case. The participants were 
prompted with self-reflection questions based on their group status (experimental versus control). The participants 
submitted their self-reflections in the assignment area of the LMS. The same procedure for the solution reports was 
used to collect the self-reflections. 

Measurement and Treatment Material 

The problem cases represented the type of ill-structured problems that would be present within the domain 
of the introductory course and were developed by the researcher. The problem cases were true problems of the 
domain, challenging, engaging but not mega-cases that tried to include every problem within the domain (Jonassen, 
1997). The course faculty then evaluated the problem cases, provided feedback on detail and relevance of the 
concepts which were then incorporated by the researcher. (See appendix A.) 

The question prompts developed for the study were based on the process of question prompt development 
of Ge (2001), Ge and Land (2003), and Ge et al. (2010). in these studies, the question prompts were elicited from 
course faculty and tightly categorized based on the four steps of the problem-solving process: problem 
representation, developing solutions, making justifications, and monitoring and evaluation. The question prompts 
were domain specific and designed to stimulate deep levels of comprehension by asking what, how, why, and which 
that can help the participants reflect on their prior knowledge (Jonassen, 1997). The prompts were a combination of 
problem-solving prompts, metacognitive prompts, and knowledge integration prompts. (See appendix B.) 

The experts who were chosen for presenting the alternative perspectives had specific knowledge or 
experience about the problem cases. This allowed participants insight into different viewpoints that could impact the 
case. The experts were videoed as they worked through the problem cases and discussed their view of the problems, 
needed solutions, and evidence for their decisions. 

The participants submitted solution reports which were using a scoring rubric. The rubric was based on an 
analytical rubric system that was developed by Ge (2001) and based on performance criteria. As a framework for her 
rubric, Ge (2001) also referred to literature by Barnes (1994) and Blum and Arter (1996) and to the rubrics 
developed by Hong (1998), who validated her rubrics through construct validity. 

For the present study, the ISP constructs and the attributes for each ISP construct were used as developed 
and validated by Ge (2001). Examples or criteria for each construct were developed based on the same ill-structured 
tasks Ge (2001) used in her study. Based on correspondence with Dr. Ge, the examples or criteria for each construct 
of the rubric for the presents study were developed to be specific to each of the problem cases within the study. 

Results 

All participants completed a baseline problem case without scaffolding support. Scores from their problem 
solution reports were analyzed using a one-way Analysis of Variance (ANOVA). The ANOVA showed that the 
baseline test scores for each of the dependent variables across groups were different on the constructs: making 
justifications and monitoring and evaluation. Baseline test scores did not differ on representing the problem or 
developing solutions. Post hoc analysis revealed that there were significant differences between the Question 
Prompts group and the Alternative Perspectives group in making justifications (p = 0.046) and monitoring and 
evaluation (p < 0.001). Table 1 is a summary of the means and standard deviations for the four dependent variables 
across all groups. 

Table 1. Score? at Baseline Across Groups 


Scores 

Control 
Mean (SD) 

Question 
Prompts 
Mean (SD) 

Alternative 
Perspective 
Mean (SD) 

F 

P 

Problem 

4.07 (1.492) 

3.93 (1.386) 

3.57 (1.441) 

0.662 (df=2,62) 

0.519 

Solution 

4.29 (1.069) 

4.43 (1.2) 

4.13 (1.359) 

0.369 (df=2,62) 

0.693 

Justify 

3.5 (1.286) 

3.36 (1.201) 

2.61 (1.196) 

3.775 (df=2,62) 

0.028 

Monitor 

1.14(1.512) 

1.64(1.129) 

0.39 (0.722) 

8.127 (df=2,62) 

<0.001 


Note. SD = standard deviation; F — F statistical value; P = degree of statistical significance 
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Due to the significant differences in the baseline of the constructs of making justifications and monitoring 
and evaluation , the baseline scores were used as a covariate in the repeated measures statistical analysis for the 
research question. 

Results from the statistical analysis results showed that scaffolding using alternative perspectives 
had a significant effect on problem representation in both Case 1 and Case 2. However, scaffolding using 
question prompts only had a significant effect in Case 2. There was no significant difference between 
groups on the construct solution development in relation to either case, while the control group and those in 
the Question Prompts group performed at an equivalent level while constructing arguments and providing 
evidence. The Alternative Perspectives group performed at a much lower level than the other two groups. 

The statistical analysis for the construct of monitoring and evaluation showed the question prompts as 
supportive but the alternative perspectives as not supportive for both Case 1 and 2. 

During participant verbalizations, listening to alternative perspectives or using question prompts 
supported the participants in representing the problem in more detail when compared to those who did not 
use scaffolding; and, listening to alternative perspectives supported participants’ ability to develop solutions 
and provide higher quality ; solutions compared to those not using scaffolding. Thinking aloud using 
question prompts supported one participant in her ability to do solution development. There is no evidence 
that the other participant used the prompts or that they affected her solution development. Also, the 
Alternative Perspectives participants constructed more arguments compared to those with question prompts 
and those without scaffolding. Finally, participants in the Question Prompts group provided evidence for 
their arguments which was not seen when analyzing verbal responses from the Alternative Perspectives 
group or one participant in the Control group. 


Discussion 

The findings of the study did not provide a consistent picture of the effect of the question prompts or the 
alternative perspectives in supporting novice nursing students during ill- structured problem-solving. There are 
several reasons why this study may have shown inconsistent results. For example, Ge and Land (2004) found if 
students were unable to interpret the prompts, this led to participants to struggle or dismiss the purpose of the 
prompts and not use them. In the reflections, an average of 20 percent of the participants stated that the question 
prompts did not help them and at times made the process more confusing. 

In Ge, Chen, and Davis (2005) they found that participants who had little experience with problem solving, 
being required to use the question prompts was effective, but for students with more experience, the students felt the 
prompts interrupted their flow of thoughts. When experts solve novel problems, previous schemas can be activated. 
The presence of additional guidance can cause increased load on working memory and thereby decrease 
effectiveness and use of guidance technique, a phenomenon called expertise reversal effect (Kalyuga, Ayres, & 
Chandler, 2003). 

Even though all participants in the present study were in their first nursing courses, data on the 
level of problem-solving experience of the participants was not collected for the study. An explanation for 
the prompts not being used or causing confusion could be due to the wide range of ages of the participants 
and different levels of experience in relation to problem-solving. This could have caused some of the 
students to disregard the question prompts. 

The effect of participant bias as well as the lack of integration of domain knowledge can also impact the 
effectiveness of question prompts. Some of the participants commented in their reflections that the question prompts 
“were not really helpful” and “I knew my own thinking process for why I came up with the solutions”. This could 
have caused the participants to simply overlook the question prompts. The lack of domain knowledge would prevent 
the participants from activating previous schema which is needed to successfully solve problems (Ge et ah, 2005). 
The problem cases were based on previous content taught in the course as well as additional information and 
resources that were provided to supplement the content. Participants ‘self-reflections revealed that many did not 
access additional resources. It also is possible that participants did not integrate domain knowledge from the earlier 
in the course. 

In two studies where alternative perspective or experts were used to model the problem-solving process, 
participants either had unlimited access to the videos or had discussions with the experts in an asynchronous 
environment (ChanLin & Chan, 2007; Choi & Lee, 2009). The ability to view the experts discussing the problems 
multiple times and the ability to ask the experts questions increased the ability of the participants to internalize the 
problem-solving process as well as reflect on knowledge and learning (ChanLin & Chan, 2007; Choi & Lee, 2009). 
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During the present study, due to issues with the link to the alternative perspectives video, the participants could only 
view the experts one time. This decreased their ability to evaluate and integrate the problem-solving process 
undertaken by the experts and could have affected the overall effect of the expert modeling. Other study findings 
also suggest that in order for experts or alternative perspectives to affect problem-solving abilities, there is a need for 
dialogue and interaction which can increase the chance of integration of domain knowledge which is crucial to the 
problem-solving process. 

Another finding by Ge et al. (2005) was the difference in the verbalizations of the participants and their 
written solution reports. Ge and her colleagues found that the solution reports did not completely reflect the level of 
reasoning demonstrated during the think-aloud protocols. Findings from the present study align with these findings: 
the problem solution reports did not always reflect the level of reasoning found during the participants’ 
verbalizations. When comparing verbalizations with the participants’ problem solution reports, there were instances 
in the current study, where identification of the problem, the cause and effect of the problem, seeking additional 
information, sub-goals, and arguments to support the solutions were not evident in the problem solution report but 
were evident in the verbalizations. Based on this finding, it is possible that the problem solution reports of those not 
videoed also may not have captured all of the participants/ reasoning and thoughts. This would have especially 
impacted results related to question prompting. 

Further research on developing learning environments to promote and facilitate problem solving is in great 
need. Successful problem solving is a skill set needed by most people in both their everyday and working lives. The 
analysis of the findings from this research study provides evidence that there are many variables that affect how 
scaffolding support can affect problem-solving abilities and that there is still much to be learned about these 
processes and impact on reasoning for nursing professionals. 
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APPENDIX A: INTERVENTION PROBLEM CASE 1 

The county where you live has been identified in a state report as having higher levels of heart 
disease and diabetes than the national average. The superintendent of the three high schools within the 
county is concerned for the future health of the children in her schools and would like to institute programs 
to increase their health and decrease their risk factors. Many of the county residents as well as the students 
are overweight or obese. The county has a diverse culture base with families who are African American, 

Hispanic, Caucasian, and Asian. Many of the families in the county get some type of government 
assistance. Funding for many of the after school programs have been cut as well as there has been a 
decrease in the amount physical education included in the school day due to trying to address learning 
outcomes required by the state. As the nurse for the three high schools in the county you have been asked to 
develop programs of some type to address this issue and then present your plan to the superintendent. 

Task 

Your task is to analyze the problem, propose solutions, support your solutions with evidence, and evaluate your 
solutions. Your final submission will be a solution report that details your identification of the problem and solutions 
with justifications for your solutions. 


APPENDIX B: PROBLEM-SOLVING QUESTION PROMPTS 


1. Define the problem 

■ What facts from this case suggest a problem? 

■ What do you already know about the problem? 

■ Do you need additional facts to define the cause(s) of the problem? 

■ What is (are) the probable cause(s) of the problem? 

2. List and Evaluate alternative solutions 

■ List two alternatives to solve the problem 

■ Evaluate each alternative by describing it advantages and disadvantages. 

3. Choose, justify, and implement a plan 

■ Which option will you implement as a plan? 

■ Why is this plan the best choice? 

■ How will you implement this plan? 

4. Evaluate the plan 

■ How and when will you monitor the implementation of the plan? 

■ How will you know if the problem is solved, alleviated, or is getting worse? 

■ What secondary problems should you watch for, and how would you do it? 
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“Technology . . . consists of more than structures and machines alone, more than just ‘hardware.’ It includes the uses 
of those structures and machine in the organization, evolution, and sometimes destruction of society” (Segal, 1994, 
p.2). Historian Howard Segal’s suggestion that technology developments are a mixed blessing is profound when one 
considers the phenomenon of cyberbullying. The plethora of affordable technologies, used by Millennial, enhances 
the need for exploration into how they are used to bully others. This study evaluates the currently accepted definition 
of cyberbullying. 

College undergraduates walk a line between the immature behavior of secondary school and their emerging 
adulthood. While some research indicated that cyberbullying is most severe in middle school and decreased during 
secondary school (Raskauskas & Stoltz, 2007; Wolak Mitchell, & Finkelhor, 2007; Williams & Guerra, 2007), it is 
also evident that the college environment is not immune to cyberbullying (Englander, Mills, & McCoy 2009; Finn, 
2004; Walker, Sockman, & Koehn, 2010). 

The Current Understanding of Cyberbullying 

A comprehensive literature review of cyberbullying research, focused on adolescents and young adults, 
revealed that there is little agreement regarding the wording and incidents that qualify as a bullying event propagated 
via technology. Most researchers have merely expanded Olweus’ definition to include technology (Burnham et ah, 
2011; Leenaars & Rinalid, 2010; Wright et ah, 2009). Li (2006) considered cyberbullying to be a “bullying problem 
occurring in new territory” (p. 166). Slonje and Smith (2008) expanded Olweus’ concept of bullying to define 
cyberbullying as aggression that utilized modern technology specifically the World Wide Web and cell phones. 

Spears. Slee, Owens, and Johnson (2009) expressed concern regarding the definition of cyberbullying via 
their qualitative research with twenty students (aged 12 - 18) and six school counselors in Australia. The authors 
acknowledged the repetitive nature of Olweus’ definition of bullying as having “common agreement” (p. 153) 
amongst researchers, yet questioned what the actual concept of repetition involved in the new atmosphere of 
cyberspace. Students interviewed considered cyberbullying to be something via technology that was used to 
intimidate or put down another. They described cyberbullying based on its emotional impact with it “sounding cruel, 
vicious, obscene, torturous, powerful and even silent” (p. 192). Those cyberbullied stated it “felt unnerving, 
demeaning, inescapable, unsafe, vulnerable, and trapped within a huge power imbalance” (pp. 192 - 193). 

Repetition was implied via plural responses but not specifically indicated as a necessity to inflict pain. 

Other researchers moved away from Olweus and conducted research based on the concepts of harmful or 
cruel events to provide the different conceptualizations and create a common language (Abbott, 2011; Vandebosch 
& Van Cleemput, 2008). Terms such as aggressive, intentional, repetitive, willful and repeated, defamatory, and 
hostile are frequently utilized by researchers working to understand the impact of negative communications 
(Englander, et ah, 2009; Hoff & Mitchell, 2009; Johnson, 2011; Smith, Mahdavi, & Carvalho, 2008). 

The reported extent of cyberbullying victims varies greatly in percentages, despite similarities in other 
demographics such as age, location, and gender. These vaiying results have increased the necessity for an 
operational definition for cyberbullying that will be universally accepted and provide more standardized reporting 
from victims and bullies alike (Abbott, 2011; Vandebosch & Van Cleemput, 2008). 
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The Emotional Toll of Cyber bullying 


The same negative emotions experienced with cyberbullying by teenagers were also reported for young 
adults. Reports of cyberbullying victimization for college-aged individuals ranged widely from eight to fifty-six 
percent. The range of those who were cyberbullies was from three to 20 percent. The range of percentages reported 
for cyberbullying may be due to lack of consistency in data gathered due to a non-standardized definition. One goal 
of this research is to operationalize the event called “cyberbullying.” 

The tragic suicide death of Tyler dementi, on September 22, 2010, catapulted the discussion of college 
level Internet victimization and suicide into the mainstream media (Cloud, 2010). Schenk (2011) was the only 
research article to report on suicide attempts or thoughts. The depth of depression that some victims feel when 
cyberbullied indicates the necessity for more research to better understand the impact of the proliferation of social 
media accessibility for college aged individuals. 

Cyberbullying on the College Campus 

When young adults leave their homes and enter college, they do so with mixed emotions of trepidation and 
excitement. Venturing onto the college campus with great expectations of good things to come may leave them 
vulnerable to the unexpected negativity that Internet and cell phone harassment can generate. Although many may 
believe that cyberbullying was left behind with high school days, Chapell et al. (2006) found that over half of 
respondents who admitted to being bullies in high school also bullied others at college. Although there is a strong 
body of empirical literature addressing bullying and a growing amount of research to understand the impact of 
cyberbullying on teenagers, research on the college level remains limited (Coleyshaw, 2010). 

Concern “prompted by shock at the dementi suicide and the increasing reports of incidents on college 
campuses” motivated the research of Baldasare, Bauman, Goldman, and Robie (2012, p. 130) which provided 
qualitative data regarding students understanding of the phenomenon of cyberbullying. The authors analyzed 
information provided by 30 undergraduate college students (M age = 20.47, SD = 2.3) to uncover major themes of 
student understanding of the events of cyberbullying. 

Findings indicated that participants were divided in the concept of intent with cyberbullying. “Many 
participants” (p. 136) indicated that harm may occur unintentionally when a receiver is hurt via messages sent with 
technology. However, “more participants” (Baldasare et al., 2012, p. 137) replied that the receiver’s interpretation of 
the event was the defining factor with one student stating, “I think maybe the definition needs to capture, like, really 
emphasize the way the recipient feels, not necessarily the way the person intended it” (Baldasare et al., 2012, p. 

137). In addition, participants considered the ability to post anonymously as a factor in cyberbullying, noting that the 
lack of face-to-face interaction removed the personal factor, “It’s almost like bullying a machine, so it doesn’t 
matter” (Baldasare et al., 2012, p. 138). Finally, the respondents consistently identified women as being more 
involved in cyberbullying than men. 

Kenworthy, Brand, and Bartrum (2012) provided a service-learning platform to educate undergraduate 
college student regarding the incidents and impact of cyberbullying. From September 2010 to January 2011, 
undergraduate students created informative presentations for over 10,000 students in secondary schools. Utilizing 
the definition of cyberbullying as “a method of bullying using technology ... to bully verbally, socially, or 
psychologically” (p. 86) the authors guided 77 undergraduate students as they researched, designed and performed 
presentations for secondary students (N=331). 

Pre and post-program data from the secondary students indicated a significant difference in their 
knowledge accuracy regarding bullying and in behavior changes they would adopt to reduce bullying. In addition, 
the university students indicated a strong value gained from the program. Responses to a post-presentation 
questionnaire indicated that while 44% had never thought about cyberbullying issues before, 86% noted that the 
experience would change their online behavior. 

Schenk (20 1 1 ) provided an experiment analysis to determine the psychological impact of cyberbullying 
victimization for college students. Participants (N = 799) ranged in age from 18-24 (M age =20.01, SD=2.41). This 
sample was further divided into group of “victims” (replied yes to having experienced cyberbullying at least four 
times) and a control group («=69). No significant differences were found in the demographics between experimental 
and control groups. 

Results indicated victims of cyberbullying were significantly higher in depression, anxiety, phobic anxiety, 
and paranoid subscales. In addition, 5.7% of victims (/?= 4) reported attempting suicide (control = 0%) and 10.1% 
had frequent suicidal ideations (control = 0%). Interestingly, phone calls were the most prevalent media of 
victimization (80%) of the five measured (text messaging, Internet, picture/video messaging, and masquerading). 
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The most common attack for both genders was an attack on their self-worth. The victims of cyberbullying were 
likely to victimize others. 

An investigation of the use of technology to cyberbully was conducted by Walker, Sockman, and Koehn 
(2011). Students (N=131) were surveyed and results indicated that up to 54% of respondents knew someone who 
had been cyberbullied. One hundred percent of male participants knew someone who had been cyberbullied. Eleven 
percent (n= 14) had been cyberbullied, with 14% of those (n—. 2) having been bullied over ten times. 

Smith, Grimm, Lombard, and Wolfe (2012) surveyed 340 undergraduate students. Results indicated that 
37% of respondents knew someone who had been cyberbullied, 3% («=10) admitted to being a cyberbully and 
16.7% were the target of cyberbullying. Statistical significance was noted in four areas. Students who self-identified 
as being a member of a Greek society were significantly more likely to observe someone they knew being targeted 
by cyberbullying. A significant relationship was also noted with college living arrangements; students living off- 
campus were more likely to know someone who had been cyberbullied than on-campus individuals. Though not 
hypothesized, the authors noted statistically significance in the response of female and non-heterosexual students in 
being more likely to know someone who had been cyberbullied in college. 

Theoretical Perspective of Social Dominance Theory 

Social Dominance Theory explains the impact of the social order of dominant attitudes based on society at 
large (Creswell, 2009; Sidanius & Pratto, 1999). The “theoretical Catholicism” of the Social Dominance theory can 
be utilized to understand the aspects of social actions that range from “acts of mobbing in the playground, to mild 
forms of prejudice and street gang violence.” (Pratto, Sidanius, & Levin, 2006, p. 57). 

Human society provides an inherent group-based structure in which dominant and subordinate hierarchies 
reside. These hierarchies are established and maintained through legitimizing myths. In society, hierarchy-enhancing 
myths (HE myths) are those noted to support the inequality that is inherent to group-based hierarchical system. 

Often focused on individual responsibility, the Protestant work ethic, and political conservatism, individuals who 
support HE myths believe that position in society was earned and therefore deserved. Contrary to HE myths are the 
hierarchy-attenuating myths (HA myths). HA myths are noted to support equality and are based in the major themes 
of socialism, feminism, and universal rights of mankind (Pratto et al., 2006; Pratto et al., 1994). To determine the 
extent that each individual accepts or applies HE and HA myths to their daily lives, one must evaluate the concept of 
Social Dominance Ordinance (SDO). 

SDO is the measure of an individual’s general willingness towards endorsing legitimizing myths to support 
or deny group-based hierarchies. The behavior of individuals is connected to the levels of social power of each 
group. 


Social Dominance Ordinance and Gender 

Research has indicated a relationship between SDO and gender. Studies have indicated a significant 
difference in the SDO level between genders, with men having a higher SDO than women (Caricati, 2007; 
Dambran, Duarte, & Guimond, 2004; Foels & Reid, 2010; Pratto et al., 2006; Pratto, et al., 1994; Zakrisson, 2008). 
These findings prompt this researcher to hypothesize a higher level of victimization from men than women. 

Social Dominance Ordinance and College Major 

Pratto et al. (1994) provided data to better understand what academic interests individuals had and 
correlated those findings to SDO levels. Eleven samples, collected from the spring 1990 through April 1992, 
provided data from 1, 747 college students from universities in the United States. Men scored significantly higher 
than women in SDO levels. 

In addition, the researchers queried the respondents’ post-college career preference. Using a theoretical 
basis, 20 career choices were provided and demarcated as hierarchy enhancing (HE) or hierarchy attenuating (HA) 
(p. 747). Results supported the hypothesis that participants who planned HA careers were lower in SDO levels than 
those interested in HE career paths, even after controlling for gender. Research results have indicated a correlation 
between college major, gender, and SDO. An analysis of the relationship between college major and cyberbullying 
is imperative. 
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Method 


In keeping with the representative research that has addressed cyberbullying on the college campus a 
descriptive study was conducted utilizing a survey instrument (Cronbach’s a = .761). 

Although the term cyberbullying was not utilized for the majority of the data gathered to prevent participant 
self-selection bias (Akbulut & Eristi, 2011; Juvonen & Gross, 2008), it was provided for the final question in the 
survey when the definition of Walker et al. (2011) defined cyberbullying as: 

The use of interactive technologies such as social networking sites, cell phones (text, video, voice, or 
picture messaging), instant messaging, or other newly developed technology-based communication tools. 
These tools are used to deliberately and repeatedly deliver slanderous, harassing, obsessive, or obscene 
messages that result in harm to the recipient (p. 37). 

Sample, Population, and Participants 

A random, cross-sectional sample for this research was drawn from the population of college students at a 
mid-size public university. Following IRB approval, data collection was facilitated via Qualtrics™ via the college 
email service. Seventy percent of participants (N= 438) were female and 30% were male. Participants were limited 
to individuals between the ages of 18 - 24. To assure that this study was adequately able to detect effect, a priori 
power analysis was conducted. 

Cohen’s (1988) tables indicated the necessity for a sample of over 200 to analyze correlation data and 64 
participants for each group in the independent samples t test to achieve an 80% probability of detecting a real effect 
(20% probability of Type II error) with a medium effect size (.30). A sample of 370 respondents was determined to 
generalize findings to the campus population of approximately 1 1,000 undergraduate students (Patten, 2009). When 
two groups differ in terms of sample size, the harmonic mean of the two is used to determine whether the a priori 
per-group sample size for t tests has been met. Each measure exceeded the predetermined participant size with the 
exception of the cyberbully variable, which was within acceptable limits (M H = 57.6). 

Results 

Data analysis consisted of descriptive statistics, coefficient correlations, and independent samples t tests. 
Crosstabulation data were analyzed to indicate the level of emotional response reported by respondents who had 
experienced the fourteen cyberbullying items only one time (See Table 1). The percentage of individuals reporting 
feeling moderately to extremely hurt after only one incident ranges from 25 to 89 (See Table 1). 
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Table 1 


Crosstabulation of Respondents Moderately to Extremely Hurt, Angry, or Sad After Being Cyberbullied One Time 
l\ 438) 


Scale 

Hurt/Sad 

Angry 

Scared 

Total % 

Total n 

Received Unwanted, Inappropriate 

% 

7.8 

12.2 

8.6 

29 

37 

Messages 

N 

130 

130 

130 



Received Unwanted, Pornographic 

% 

9.5 

24.3 

13.7 

48 

31 

Images 

N 

66 

66 

65 



Replied Unknowingly to Someone 

% 

10.0 

18.2 

8.5 

37 

22 

Posing as Someone Else 

N 

60 

61 

59 



Facebook Friend “Friended” for 

% 

8.8 

26.7 

11 

47 

42 

Information 

N 

91 

90 

90 



Received Harassing or Threatening 

% 

21.1 

39.7 

15.3 

76 

56 

Messages 

N 

71 

73 

72 



Teased Due to Physical Appearance, 

% 

23.1 

28.5 

4.8 

56 

37 

Personality or Intelligence 

N 

65 

63 

62 



Harassed Due to Sexuality 

% 

25 

0 

0 

25 

1 


N 

4 

4 

4 



Target of Untrue Gossip or 

% 

36.3 

43.2 

9.1 

89 

39 

Humiliating Comments 

N 

44 

44 

44 



Had Problems Due to Personal 

% 

34.3 

41.5 

11.7 

88 

61 

Information Shared w/o Consent 

N 

70 

70 

68 



“Outed” 

% 

40 

5 

0 

40 

2 


N 

5 

5 

5 



Blocked Online 

% 

26.7 

14.6 

8.2 

50 

23 


N 

75 

75 

74 



Private, Personal Images Shared w/o 

% 

11.1 

19.6 

6.6 

37 

17 

Consent 

N 

45 

46 

45 



Other People Used Your Identity 

% 

10.2 

24.1 

13.3 

48 

14 

w/o Consent 

N 

29 

29 

'30 



Been Cyberbullied 

% 

30.2 

38.1 

11.7 

80 

34 


N 

42 

42 

43 




The percentage of respondents selecting occurrences of one or more times for the thirteen relational 
bullying questions (RBS) questions ranged from 1.1 to 29.7%. When provided with the definition of cyberbullying 
as: “Social media and /or cell phones, used to deliberately and repeatedly deliver slanderous, harassing, obsessive or 
obscene messages that result in harm to the recipient,” only 9.9% of respondents selected occurrences of one or 
more times. 

An analysis of the correlation between the response rates of the thirteen RBS questions to the direct 
question “have you been cyberbullied”(CBR) is also an important aspect of determining the accuracy of the current 
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definition (See Table 3). If respondents answer yes to any of the thirteen questions, they should also answer yes to 
the “been cyberbullied” question. 

Table 3 

Correlation Between Being Cyberbullied Response (CBR) With Relational 
Cyberbullying Scales (RBS) (N=403) 



Pearson R 

Spearman Rho 

Received Unwanted, Inappropriate Messages 

. 212 *** 

.224*** 

Received Unwanted, Pornographic Images 

230*** 

264*** 

Replied Unknowingly to Someone Posing as Someone Else 

. 200 *** 


Facebook Friend “Friended” for Information 

23i*** 

229 *** 

Received Harassing or Threatening Messages 

. 504 *** 

. 429 *** 

Teased or Made Fun of Due to Physical Appearance, Personality or 
Intelligence 

. 414 *** 

. 311 *** 

Harassed Due to Sexuality 

.099* 

243 *** 

Target of Untrue Gossip or Humiliating Comments 

. 483 *** 

329 *** 

Had Problems Due to Personal Information Shared w/o Consent 

191*** 

.134** 

“Outed” 

215 *** 

25i*** 

Blocked by others 

.068 

.086 

Private, Personal Images Shared w/o Consent 

.157** 

.163** 

Other People Used Your Identity w/o Consent 

249 *** 

219 *** 

* p < .05 

**p < .01 
***p < .001 

Interpretation Guidelines: Small r=,10 to .29; Medium r=,30 to .49; 
Large r=,50 to 1.0 

(Cohen, 1988. pp 79-81). Medium and High results in bold. 


Two supporting hypotheses were posed in relation to connection of the Social Dominance Theory and a 
person’s desire to dominate others via social media and other technologies. 

Hi: College-aged men will report a higher level of cyberbullying others than will college-aged women. 

When responses were reviewed utilizing a crosstabs analysis 6.9% of female respondents (n = 20) had 
cyberbullied others one or more times compared to male respondents at 9.6% (n = 1 1). An independent samples t 
test was conducted to compare the cyberbullying occurrences for males and females. There was no significant 
difference in scores for males (M = 1.10, SD = .3) and females (M = 1.07, SD = .25; t (401) = .925, p = .36. These 
data results indicate that the concept of SDO, or the need to be dominant, is not supported for gender. 
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H ; : College students enrolled in HE majors will report a higher level of cyberbullying others than those in HA 
majors. 


A second method for delineating the impact of SDO on cyberbullying is presented via an analysis of SDO 
correlated with college major. An independent samples t test was analyzed to determine the difference of major and 
amount of self-reported cyberbullying behaviors. There was no significant difference in scores for HE Majors (M = 
1.08, SD = .3) and HA Majors (M = 1.08, SD = .03; t (384) = -.020 , p = .98. These data results signified that the 
concept of SDO, or the need to be dominant, was not supported. 

Discussion 

This chapter is approached with all of the reverence and concern necessary when one is dealing with the 
emotions of youth. The following discussion is viewed through the lens of the researcher, a social media and 
communications expert. 

The research question focused on the necessity for a re-definition of cyberbullying based in empirical data. 
Much of past cyberbullying research on the undergraduate campus utilized the parameters created by Olweus that 
established traditional bullying as negative actions that are intentional and repeated to inflict harm on another person 
(1986, 1991, as stated in Olweus, 1993, p. 9). Researchers have called into question this definition and this author 
concurs (Abbott, 2011; Vandebosch & Van Cleemput, 2008). 

Up to thirty percent of respondents ( n = 130) had experienced cyberbullying at least once via web-based 
communications and hand-held technologies. The impact of cyberbullying only one time is widespread. Young 
adults, on the college campus need the support of administration, counselors, and educators based on the emotional 
impact the cyberbullying has on them - not dependent on the act being repeated. 

When are negative, hurtful, threatening or mean comments hosted on the Internet or sent via hand-held 
technology cyberbullying? When one reviews the results noted in this study, it is impossible to overlook the 25 to 89 
percent of respondents (Table 1) who experienced feeling hurt, sad, angry, or scared after being cyberbullied one 
time. Whether being stalked via Facebook friends or sent harassing, threatening messages participants are feeling 
emotional upset due to these events. Respondents who report feeling “very hurt , ” “scared, ” or those who have an 
incident that “ruined my trust ... “ must be heard and understood. 

While not the desire of this author to use emotionally charged events, it seems one would be remiss in not 
presenting the data available from the death of Tyler dementi. In August 2010, two young men left home to step on 
the college campus for the first time, to forever have their lives changed. Though we will never know the thoughts 
of Tyler dementi in detail, his suicide shortly after learning that Damn Ravi used his webcam to spy on dementi 
speaks volumes (Sloan, 2012). 

This single incident, on September 19, 2010, changed both lives forever, dementi asked Ravi to leave their 
dorm room that evening when dementi was entertaining a male friend. Ravi remotely accessed the webcam on his 
computer, left in the room, to view dementi and his guest. Ravi was interviewed on abc20/20 and expressed his 
thoughts regarding the incident. Following the webcam viewing, Ravi tweeted to all of his friends. Ravi’s response, 
when asked why he sent it is profound, “I wanted to let all my friends from back home know. In my head, that’ s just 
how Twitter [sic] - we can still all be involved with each other’s life.” Ravi stated, “what most people type is 
wrapped with seven layers of sarcasm and another layer of irony.” Whether Ravi intended to be a cyberbully or not, 
two lives have been forever changed (Sloan, 2012). 

The author acknowledges that cyberbullying may be an intentional act, deliberately conducted to hurt or 
scare the recipient. However, these findings indicate that the current use of the traditional bullying definition 
(Olweus, 1993) to understand the impact of bullying behaviors in cyberspace is not comprehensive. This researcher 
feels that the necessity to re-define cyberbullying is at the forefront of concern to assure that all acts of 
cyberbullying; whether intentional and repeated or a single, random event; are recognized. This will allow victims to 
advocate for themselves and receive the help necessary. 

This concern is supported by qualitative research conducted by Baldasare et al. (2012) with most 
participants stating that the receiver’s interpretation of the event should provide the defining factor. Spears et al. 
(2009) utilized triangulation of qualitative data to evaluate the human dimensions of cyberbullying. Participants 
reported cyberbullying as looking like “ostracism, exclusion, and intimidation” (p. 192) and sounding “cruel, 
vicious, obscene, torturous, and powerful” (p. 192). In addition, cyberbullying felt “unnerving, demeaning, 
inescapable, and unsafe” (p. 193). Vandesbosch and Van Cleemput (2008) also utilized qualitative research with 53 
focus groups. When asked to define cyberbullying, the participants noted events such as spreading personal 
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conversations, gossip, manipulating and sending personal pictures, sending messages with sexual comments, or 
humiliating someone online. 

This study garnered expressions of emotions that included embarrassment, creepy, scary, stalkerish, 
derogatory, racist, anger, hurt, and frustrating. Therefore, the following definition is proffered: 

Cyberbullying is the use of web-based communication media or hand-held technologies by an individual or 
group to deliver slanderous, harassing, demeaning, obscene, racist or other offensive messages, images, or 
video either directly or indirectly that result in emotional harm to the target of the communication. 

Conclusion 

As society moves forward, it is through the education of our youth regarding these new communication 
challenges and how to deal with them that the future will be improved. Cyberbullying is not old wine in a new 
bottle. It is a new challenge that must be addressed as such with a new definition and education for Millennials. 

One method proposed by this author would be through the use of college curriculum to provide a 
required, hybrid college communication course to address communicating with technology and doing so with 
decency. This course would encompass interpersonal, intercultural, and social media communication theories to 
provide a basis for the orientation and integration of social media ethics and etiquette in curriculum, lifestyle, and in 
business and career. 

Research conducted by Ken worthy et al. (2102) provided the second consideration. A service-learning 
platform, utilized to educate undergraduate college students while working with secondary students, to advance their 
knowledge of how to recognize, avoid, and address cyberbullying should be considered as a vital part of the 
undergraduate college experience. 

In closing, this author sincerely hopes that all who read this study benefit. No more powerful words can 
be reiterated than those of Ravi, “1 just wish I had talked to him more . . . “ (Sloan, 2012). 
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Chinese Learners’ Attitudes towards Chat Partners in Computer-Mediated 

Communication 

Yanlin Wang, Stefania Borst, Jinglan Feng, Rong Chang, Steven Crooks 


Introduction 

Computer-mediated communication (CMC), which is increasingly being used in second language (L2) 
teaching, provides an ideal platform for language learners to engage in real-time interaction in an authentic manner 
(Smith, Alvarez-Torres & Zhao, 2003; Warschauer, 1997). Many positive effects of CMC on L2 learning have been 
reported (Liang, 2010; Pellettieri, 2000; Varonis & Gass, 1985). These include increasing both input and output of 
the target language, stimulating the production of more negotiations (Pellettieri, 2000). 

Several studies have investigated the role of chat partners in CMC, specifically looking at native speakers 
(NS) vs. non-native speakers (NNS). These studies (Chen, 2009; Freiermuth, 2001; Lee, 1997, 2009; Tudini, 2003) 
employed a variety of CMC tools (e.g., voice conferencing, blogging, and instant messaging) and found that learners 
in NS-NNS pairs learn more effectively than learners in NNS-NNS pairs in a variety of online chat formats. The 
current study fills this gap in the literature by investigating students’ attitudes toward NS vs. NNS chat partners. 

Literature Review 

Computer-mediated communication (CMC) involves a synchronous or asynchronous “conversation” that 
takes place using a computer (e.g., desktop, laptop, or tablet) (Baralt & Gurzynski-Weiss, 201 1, p. 206). CMC has 
become a popular tool in language instruction. In fact, it has been referred to as a ‘conversation in slow motion’ 
offering affective, linguistic, and even cognitive advantages, because it gives language learners more time to process 
input, plan and modify their output (Beauvois, 1992). CMC also gives language learners a platform to use their 
second language (L2) to interact with others without geographical limitations (Pellettieri, 2000). Finally, CMC has 
been found to benefit L2 learners through increased motivation and more positive attitudes towards language 
practice (Abrams, 2011). 

Given the popularity of CMC in L2 instruction, the lack of research related to chat partner characteristics is 
somewhat surprising. Abrams (2011) recognized the lack of empirical studies in this area and identified research on 
“different chat partners” with “non-Latin-based orthographies” as one of the nine most promising areas of CMC 
research. 

Group Characteristics in CMC 

CMC research in L2 learning has typically focused on interactions among students within either pairs or 
groups (three or more learners). One way of characterizing partners in CMC in these settings is by their level of 
language proficiency. For example, a partner may be either a native (NS) or a non-native speaker (NNS; includes 
most students) of the target language. Using this characterization, pairs and groups can be either NNS-NNS at the 
same or different levels, or NS-NNS. 

The primary concern with groups of NNS students working together in activities such as CMC is that 
students tend to learn incorrect language from their inexperienced peers (Satar & Ozdener, 2008). Liang (2010) 
shared this concern by stating that “synchronous online peer response groups might be fun, but not very effective” (p. 
57). Flowever, a study by Varonis and Gass (1985) suggests that the advantages of NNS-NNS groups may offset, to 
a degree, the disadvantages discussed above. 

Other studies have shown that NNS-NNS groups tend to exhibit less productive learning patterns than NS- 
NNS groups. For example, when chatting with NS, NNSs tend to provide “more elaboration, more repetition, 
slower speech, more questions, more of what may be considered linguistic correction and greater willingness to 
allow a topic shift than do native speaker responses to other native speakers” (Varonis and Gass, 1985, p. 72). Lee 
(2009) concluded that text-based chat in NS-NNS groups is a powerful mediating tool for the enrichment of 
language learning that goes beyond a traditional classroom setting. Apparently, the gap in language proficiency 
between NS and NNS students pushes the NNS’s language performance in reading, writing and speaking skills 
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(Chen, 2009; Grosbois, 2011; Kitade, 2000; Lee, 1997; Mahfouz & Ihmeideh, 2009; Tudini, 2003) as well as 
cultural understanding (Lee, 1997). 

However, negative effects fromNS-NNS language groups have also been found. Freiermuth (2001) 
compared NS-NNS online groups with NS-NNS face-to-face groups in terms of their chat turns and word 
production and found that NSs tended to dominate the group discussion, leaving the NNSs with insufficient 
opportunities to practice the target language. His study was consistent with Meskill’s (1993) finding that NSs tend to 
dominate the conversation when chatting with NNS. In the current study, we were interested in comparing the 
attitudes of the students in NNS-NNS and NS-NNS pairs specifically during CMC. 

Chinese Language and CMC 

Abrams (2011) pointed out that “most current CMC studies look at German, Spanish or ESL/EDL contexts, 
and the dearth of studies available in other languages. . .remains unsatisfying” (p. 71). However, a few studies have 
shown that CMC is feasible in Chinese language teaching and learning (Xie, 2002; Xie, 2008). Thus far, studies 
have examined: the students’ attitudes toward online chat (Lai, Zhao, & Zhang, 2009; Xie, 2002), Chinese learners’ 
oral performance (Yuan & Liu, 2010), the interactive learning in Chinese writing (Liao, 2010; Zhang, 2009) as well 
as hydride course designs (Lai, Zhao, & Wang, 2011; Xie, 2002; Zhang & Mu, 2003). All of these studies were 
conducted in online group discussion and we are aware of no research on the learners’ perception of the chat 
partners in CMC in Chinese language. 

In sum, there is no conclusive evidence to date about which chat partner can indeed help Chinese language 
learners cognitively and affectively. While previous evidence with other languages suggests that this would be 
interesting to look at, no study has methodologically compared NS/NNS Chinese chat partners with the specific goal 
of measuring attitudes. The current study aims to address this important gap in the literature. 

Research Questions 

The research question that guided this study is “What are the perceptions of the Chinese L2 learners 
towards their chat partners (NS vs. NNS) during their CMC chat ? ” 

Methodology 


Participants and Materials 

Eighteen native English speakers who are learning Chinese in the intermediate Chinese course and six in 
the advanced Chinese courses in a large public university in the southwest USA participated in this study. The 
Chinese learners received the course credit after completing this activity. Seventeen Chinese native speakers who 
are studying in the same university also participated in as volunteers to chat online in text with the Chinese learners. 
Finally 18 Chinese learners completed all activity four chats with two kinds of chat partners were analyzed. 

Abrams (2011) suggested that topics for CMC activity should be selected in consultation with the learners 
if possible because it can motivate the learners higher and make the task-completion more likely. After surveying 
the Chinese learners, the four favorite chat topics emerged are Hobbies, Entertainment, Food and Culture. 

Procedure 

The twenty four Chinese learners were randomly divided into two groups. Both groups consisted of 12 
students. All the students have the equal chances to chat with different Chinese native speakers and peers. The 
sequence of the two groups chatting with Chinese native speakers and peers was counter-balanced. All the Chinese 
learners were randomly assigned to their partners: Chinese native speakers or peers, but no repeated chat partner has 
been assigned. 

All the Chinese learners were assigned to do the four chat activities in two weeks during the fall of 201 1 in 
a computer lab in the university’s library where technology support was available. All the Chinese native speakers 
who chatted as the volunteers with the Chinese learners logged in the Facebook anywhere they preferred at the start 
of the chatting. The first chat started one month after the beginning of the semester. Each chat lasted about 50 
minutes. 
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The researchers created a group page on the Facebook and added all the participants as group members 
before the start of the study. On the day of chat activity, students logged in on the Facebook, clicked on the chat list, 
and looked for their assigned chat partners. 

After chatting twice with one type of chat partners, each Chinese learner needed to do an online survey 
aiming to investigate their attitudes towards their two chat partners. All Chinese learners chatted twice with 
different Chinese native speakers and twice with difference peers and completed two online surveys to the two types 
of chat partners. 

During the chat, the participants typed Chinese Pinyin in the Microsoft Pinyin input method and selected 
the correct Chinese characters to complete their Chinese sentences, and the sentences were sent to their interlocutor. 

Data Collection 

The data was collected from the online five-Likert point questionnaire with an open-ended question at 
www.qualtrics.com . The questionnaire consisted of demographic information. Chat Partners Attitude Scale and one 
open-ended question. The first section is about the demographic information, such as gender, ethnicity, major, 
academic classification, course, L2 background, etc. The second section is Chat Partners Attitude Scale developed 
by the researchers. The five-Likert point attitude scale with totally 14 questions consists of two subscales. The first 
subscale of cognitive perspective has 6 statements about the assistance of the Chinese learners gaining from chat 
partners in Chinese character, vocabulary, grammar, culture, error correction, and language improvement. The 
Cronbach a is .85 which showed a satisfactory reliability of the scale. The second subscale of affective perspective 
has 8 statements about the affection and feelings of the Chinese learners towards chat partner and Cronbach a 
= .75. The open-ended question asked the reason why/why not the participants like their chat partners. 

Data Analysis 

The data of SPSS document produced automatically from www.qualitrics.com where the questionnaires 
were administrated. Paired-sample t tests were conducted to evaluate whether the students were more willing to 
chatting with than with NNS from cognitive and affective perspectives by the two subscales. Three experienced 
Chinese language instructors coded the open-ended questions as individual reviewers. Three raters keep a higher 
inter agreement (Randolph’s free-margin multirater k is .96). 

Results 

Results from Chat Partners Attitude Scale and its Two Subscales 

A paired-sample t test (table 1) was conducted to evaluate the Chinese learners’ attitudes towards their two 
types of chat partners. There was no significant result found from the SPSS analysis. 

Table 1 


The Result of Sample t Test on Chat Partners Attitude Scale 



Mean 

N 

SD 

Mean 

difference 

95% Confidence Interval of 

the Difference t 

df 

Sig. 





Lower 

Upper 


Chatting with NS 

3.76 

18 

.47 

.13 

-.13 

.39 1.04 

17 

.31 

Chatting with NNS 

3.63 

18 

.39 


Although generally, the Chinese learners didn’t show preference to chat with any specific type of partners, 
the results in the subscale analyses revealed significant findings in the cognitive subscale. 

A paired-sample t test (table 2) from the cognitive subscale was conducted to evaluate whether the students 
were more willing to chat with native Chinese speakers based on their Chinese knowledge gaining during the chat. 
The results indicated that the mean chatting with native Chinese speakers (M= 3.57, SD =. 47) was significantly 
greater than the mean chatting with peers (M= 3.11, SD = .45), ? (17) = 3.31,/? < .01. The standardized effect size 
index, d, was .80. This result revealed that the Chinese learners preferred to chat with native Chinese speakers from 
the cognitive perspective. 
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Table 2 


The Result of Sample t Test on Cognitive Subscale 



Mean 

N 

SD 

Mean 

difference 

95% Confidence Interval of 

the Difference t 

df 

Sig. 





Lower 

Upper 


Chatting with NS 

3.57 

18 

.47 

.46 

.17 

.76 3.31 

17 

.004* 

Chatting with NNS 

3.11 

18 

.45 


* P < .05 two-tailed 


The cognitive subscale was intended to know whether chatting with a specific type of partner could 
facilitate Chinese learners’ Chinese language proficiency. The six statements in the subscale covered the Chinese 
characters, vocabulary expression, grammar, culture and error correction, etc. Paired sample t tests were conducted 
to scrutinize in which areas chatting with native Chinese speaker benefited the Chinese learners’ language 
proficiency (Table 3) and found significant results on statement 5 and 6. Chatting with native Chinese speakers (M = 
3.67, SD =1.03) inputs more culture items to the Chinese learners in their online chatting than with peers (M = 2.83, 
SD = .93), t (17) =.73 ,p< .05. In addition chatting with native Chinese speakers (M= 3.56, SD =.92) inputs more 
culture items to the Chinese learners in their online chatting than with peers (M= 2.78, SD = .94), t (17) =.44,/? 

< .05. 


Table 3 

The Results of Sample t Tests on Each Statements in Cognitive Subscale 


Statements 

Group 

Mean 

N 

SD 

Mean difference 

t 

df 

Sig. 

1. My chat partner taught me some 
new Chinese characters during my 
chatting. 

NS 

4.00 

18 

.97 

.56 

.89 

17 

.076 

NNS 

3.44 

18 

.98 

2. 1 learned some new 
phrases/expressions from my chat 
partner 

NS 

3.94 

18 

1.11 

.44 

.41 

17 

.177 

NNS 

3.50 

18 

1.04 

3. My chat partner didn’t help me 
much in my Chinese language 
improvement. (Reversed) 

NS 

3.83 

18 

.86 

.44 

.57 

17 

.134 

NNS 

3.39 

18 

.85 

4. 1 didn’t learn some new grammar 
items during my chatting. 
(Reversed) 

NS 

2.44 

18 

.98 

-.28 

-1.10 

17 

.288 

NNS 

2.72 

18 

1.02 

5. My chat partners showed me 
some Chinese culture items in our 
chatting 

NS 

3.67 

18 

1.03 

.83 

.73 

17 

.014* 

NNS 

2.83 

18 

.92 

6. My chat partner corrected 
mistakes in my Chinese sentences. 

NS 

3.56 

18 

.92 

.78 

.44 

17 

.026* 

NNS 

2.78 

18 

.94 


* P < .05 two-tailed 


The analysis of the affective subscale with eight statements (see table 5 for statements) didn’t show 
significant results. Statistically, the Chinese learners didn’t think there was any difference from the affective 
perspective between chatting with native Chinese speakers (M= 3.57, SD =. 62) and with their peers (M= 3.74, SD 
= .46), 1(17) = -.77, ns. The difference in the means didn’t seem to indicate that Chinese learners were more nervous 
chatting with native Chinese speakers than with peers. 

Table 4 

The Result of Sample t Test on Affective Subscale 


Mean N 


SD 


Mean 95% Confidence Interval of 
difference the Difference 


df Sig. 
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Lower 


Upper 


Chatting with NS 3.57 18 .62 

— -.17 -.62 

Chatting with NNS 3.74 18 .49 


.29 -.77 17 .45 


When we analyzed each statements in the affective subscales (Table 5), we found that the fast typing speed 
of the Chinese native speakers (M— 3.06, SD =1.11), Statement 1, made Chinese learners more nervous than 
chatting with their peers (M= 3.83, SD =. 92). 

Table 5 


The Results of Sample t Tests on Each Statements in Affective Subscale 


Statements 

Group 

Mean 

N 

SD 

Mean difference 

t 

df 

Sig. 

1 . The fast typing speed of my chat 
partner made me nervous. 
(Reversed) 

NS 

3.06 

18 

1.11 

-.78 

-2.30 

17 

.035* 

NNS 

3.83 

18 

.92 

2. 1 felt anxious when 1 thought my 
chat partner’ Chinese proficiency 
was much higher than mine. 
(Reversed) 

NS 

3.11 

18 

1.23 

-.17 

-.42 

17 

.681 

NNS 

3.28 

18 

1.13 

3. 1 was confused by most of the 
sentences made by my chat partner. 
(Reversed) 

NS 

3.33 

18 

.58 

.56 

-1.97 

17 

.066 

NNS 

3.89 

18 

.97 

4. 1 was very relaxed and 
comfortable to chat with my chat 
partner. 

NS 

3.67 

18 

.97 

.44 

-1.33 

17 

.203 

NNS 

4.11 

18 

.68 

5. Chatting with my chat partner 
was an amazing experience to me. 

NS 

3.89 

18 

.76 

.28 

1.16 

17 

.263 

NNS 

3.61 

18 

.78 





6. Sometimes, I felt it was boring to 
chat with my chat partner. 
(Reversed) 

NS 

3.78 

18 

1.00 

.22 

.59 

17 

.562 

NNS 

3.56 

18 

.92 

7. 1 would like to continue chatting 
with my chat partner in the future. 

NS 

4.06 

18 

.80 

.17 

.57 

17 

.579 

NNS 

3.89 

18 

.68 

8. Several times, during my 
chatting, I felt like to quit. 
(Reversed) 

NS 

3.67 

18 

1.03 

-.06 

1.16 

17 

.875 

NNS 

2.83 

18 

.92 


* P < .05 two-tailed 


Results from the Open-ended Question 

We also analyzed the open-ended question from the questionnaire to explore the reason why the 
participants liked/disliked to chat with a specific type of partner in the online chat. The students’ explanations could 
help us triangulate the findings from the questionnaire. 

Table 6 

The Reason why the Chinese learner like and dislike chatting with NS and NNS 

Chat with NS Chat with NNS 



Codes 

Times 

Codes 

Times 

Reasons of like 

Nice personality 

10 

Nice personality 

6 

Fun experience 

6 

Fun experience 

3 
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Positive language input 

7 

Positive language input 
Same language proficiency 

3 

1 

Crammed information 

Reasons of dislike 

1 

Simply language input 

3 

Boring activity 

1 

Unchallengeable partners 

3 


In the table 6, apparently “nice personality” is the key reason liking the chat partners, no matter whether the 
chat partners are NS or NNS. “Nice personality” of the chat partners was mentioned 10 times when the students 
answered this question after they chatted with NS, which indicated that Chinese learners’ first impression to the NS 
was the “nice” personality. “Nice personality” included the following words used to describe the chat partners: nice, 
helpful, friendly, patient, supportive, personable, pleasant, enjoyable, etc. The category of “fun experience” when 
chatting with NS included comments like “had interesting things to talk about”, “I had a pleasant conversation”, 
“nice conversation”, “talked about interesting topics and made me feel welcome”, “fun to talk to”, “very interesting 
experience that I hope to do more of’. Only 3 times was “fun experience” mentioned when chatting with NNS. 

The Chinese learners mentioned 7 times that the NS gave them “positive language input”, like “1 liked 
chatting with them because they were very good at explaining what they meant if I didn’t understand what they 
said”, “like practicing Chinese”, “they were . .. supportive . . . with my poor ability to chat in Chinese”, “I feel that it 
increases my vocabulary and helps me communicate with Chinese speakers close to my age”, “I preferred speaking 
to Chinese Natives. . . and I felt like I learned more”, etc. Only three times was this category mentioned when 
chatting with NNS. One Chinese learner did mention that “chatting with peers was much easier and enjoyable 
because even if they are ahead or behind me in learning, they understand what material we should know or is 
beyond us, and our general level of proficiency is about the same, so it is much easier to communicate.” 

As to the reasons of occasional dislike the NS, it was interesting to find that only one participant thought 
that NS input “crammed information” during their chat. The participant explained: “my second partner however 
spoke in great detail about the topics of discussion, so it was much too difficult to keep up.” and only one participant 
mentioned “boring conversation”. 

As to the reason of dislike the NNS, “simple language input” and “unchallengeable partners” were the two 
reasons and were mentioned three times for each. The participants said “they were both class mates so 1 have 
already talked to them in Chinese so it was not a challenge”, “With my peers it was more of I knew exactly what 
they were saying because we just went back and forth repeating things that I already knew how to say”, “it was fun 
to chat with them, but I didn't really learn much”, etc. 

Discussion 

The Cognitive Advantage of Chatting with NS in CMC 

The results of the Chat Partners Attitude Scale generally did not confirm that the Chinese learners preferred 
one kind of chat partners. However, the statistically significant results in the cognitive subscale revealed that the 
participants were more willing to chat with NS, because of the positive language input, especially the culture input 
and the error correction by the NS. Additionally, in the open-ended question about the reasons they liked to chat 
with NS, positive language input ranked second. The students viewed the chatting with NS as a chance to “practice 
Chinese”, increase their vocabulary, and learn more, because the NS were very “supportive” and “good at explaining 
what they meant if 1 didn’t understand what they said”. 

Our findings were consistent with the previous studies (Chen, 2009; Grosbois, 2011; Kitade, 2000; Lee, 
1997; Mahfouz & Ihmeideh, 2009; Tudini, 2003), but supported their statements from an empirical perspective. 
Negotiation of meaning is one of the important indicators to the language learners’ outcomes in the Interaction 
Hypothesis, which states that negotiating meaning (i.e., resolve their miscommunications) with other speakers, 
native or otherwise, is crucial to language acquisition (Long & Robinson, 1998). Smith (2003) claimed that 
“negotiated interaction in particular is viewed as beneficial for SLA as learners elicit modified input from one 
another, are pushed to modify their own linguistic output, and receive important feedback on their target language 
use, thus potentially focusing their attention on their problematic utterances”. 

Tudini (2003) also claimed that chatting with NS is “a valuable connection to the target language and 
culture which can provide learners with the opportunity to develop colloquial interactive language which is rarely 
found in textbooks” (p. 155). In addition. Long (1996) described that “negotiation work that triggers interactional 
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adjustments by the NS. . facilitates acquisition because it connects input, internal learner capacities, particularly 
selective attention, and output in productive ways” (pp. 451-452) 

To the Chinese learners who highly valued the chatting with NS, in addition to the “fun activity” and the 
“nice personality” of the NS, the increase in interaction with NS in Chinese language, and the discussions with more 
culture context were the key points. 

In light of these results, we strongly recommend chatting with NS in CMC. It gave the L21eamers’ error 
correction and motivated them to understand the context. 

Recommendation and Limitations 

Based on our findings, pedagogically we strongly suggested the instructors choose the NSs as the chat 
partners in CMC activity because of the benefit of language proficiency gap which scaffold the language learning 
and also NSs showed interests to chat with NS for the cognitive benefit. However, we cannot ignore that the favorite 
chat partners should have the characteristics of kind, patient, nice, helpful, interactive and supportive, thus and so 
the interaction will bring up learners’ confidence on their Chinese language proficiency. To a certain degree, we 
suggestion that when the instructor design such activity, it’s better to foresee and predict the NSs personality and 
avoid to choose the NSs who push too much and bomb the learners by the tons of sentences which excess the 
learners acceptance. 

This study showed some limitation because of the small sample. The transcripts of the chatting should be 
analyzed to check the words, negotiation turns, negotiation routines produced by the Chinese learners with 
difference chat partners in order to support our findings. 
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Abstract 

The purpose of this study is to explore inhibiting factors teachers in the United States have personally 
encountered or perceived in using games for instruction, and how teacher perceptions of barriers are influenced by 
their gender, teaching level and experience of using games for instruction. Four factors have been found to hinder 
teachers’ use of games in the classroom: challenges of implementing games effectively, challenges with using 
technology, current educational system, and challenges with obtaining games. Male teachers, more than female 
teachers, regarded challenges of implementing games effectively as a serious barrier, whereas female teachers 
perceived challenges with using technology and obtaining games as a more serious barrier. Middle/intermediate and 
high school teachers believed challenges of implementing games effectively to be a more serious barrier than 
primary school teachers, while primary school and middle/intermediate school teachers viewed challenges with 
obtaining games as a more inhibiting barrier than senior high school teachers. Teachers who have used games for 
teaching, more so than teachers who have not used games for teaching, thought that use of games is less hindered by 
challenges of implementing games effectively and current educational system. 

Introduction 

An increasing number of practitioners and researchers are advocating for video games as a promising form 
of instruction that can both engage students and strengthen skills important in the current information age (Aldrich, 
2004; Federation of American Scientists, 2006; Foreman, Gee, Flerz, Hinrichs, Prensky, & Sawyer, 2004; Prensky, 
2001; Quinn, 2005). The Federation of American Scientists (2006) has identified video games as well suited for 
educating learners in preparation for working in today’s knowledge economy, providing motivating instruction for 
the development of higher order thinking skills, expertise, teamwork, and problem solving. 

Despite the strong push for the use of educational video games and a research base stretching back decades, 
the research literature contains very few quality studies on how effective educational games are at promoting 
learning (Fletcher & Tobias, 2006; Ke, 2009). Furthermore, researchers highlight the importance of also focusing on 
the implementation of the game in the learning environment (Garris, Ahlers, & Driskell,2002; Leemkuil, de Jong, de 
Hoog, & Christopher, 2003; O'Neil, Wainess, & Baker, 2005; Wolfe, 1997), but there is limited literature examining 
how games can best be used in formal learning environments, and some researchers claim that video games are ill- 
suited to the structure of school and therefore likely do not have a place in the classroom (Squire, 2004). 

Birmingham holds that teachers in digital game -based learning (DGBL) take on many roles in terms of 
receiving training, understanding the game, guiding students and solving problems so that a teacher plays a role as 
important as the game itself in promoting students’ learning (as cited in Kirriemuir & McFarlane, 2004, p27). 
Therefore, in order to make DGBL more popular in K-12 education, it is important to conduct in-depth study on 
teachers’ perceptions of the barriers to using video games for instruction. 

Although some studies have examined teachers’ perceptions of games (Baek, 2008; Koh, Kin, Wadhwa, & 
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Lim, 2011; Pastore & Falvo, 2010; Razak, Connolly & Hainey, 2012; Schrader, Zheng & Young, 2006), they contain 
some limitations. Firstly, the previous studies either did not define the term game or failed to clarify that the games 
should have educational value as originally intended by the authors in the surveys, which could easily mislead the 
participants to relate to commercial video games. In addition, most of the previous research employed quantitative 
methods with surveys designed involving researchers’ preconceptions, which might not reflect teachers’ authentic 
and in-depth perspectives. Further, most of those studies are conducted in other countries rather than the US, which 
poses a limitation in generalization of the results to American classrooms. 

This study will try to overcome the limitations of previous studies and fill an important gap in the literature 
by gathering US K-12 teachers’ perceptions of the barriers they face in implementing video games in the classroom. 
We will survey both teachers who have used games and teachers who have not used games on the barriers they 
personally encountered or perceive. The term game will be specified as computer and video games utilized for 
educational purposes, including both commercial games with educational value and games especially designed for 
education. In terms of methodology, this study uses both qualitative and quantitative methods. The use of individual 
interviews to generate themes surrounding attitudes toward, existing barriers and challenges facing the use of 
educational video games in K-12 schools results in the development of a survey based on these themes that drives a 
large scale, quantitative study of teacher perceptions of barriers, which is similar to Baek (2008)’s study done with 
teachers in South Korea. This study will help us better understand US teachers’ perceptions of and experience with 
teaching using video games and thus contribute to the research on this topic. This study will seek to answer the 
following research questions: What barriers in using games for instruction do the teachers perceive? Do gender, 
teaching level, experience of teaching with games influence teachers’ perceptions of barriers? 

Literature Review 

Teacher Attitude Towards Using Video Games in Classroom 

Gee (2003) defines “video games” as “both games played on game platforms (such as the Sony PlayStation 
2, the Nintendo GameCube, or Microsoft’s XBox) and games played on computers” (p.l), and he gives them a 
simple name “video games” to replace “video and computer games”. This paper uses the term video games or 
games to indicate video and computer games. A number of studies have suggested that the majority of K-12 school 
teachers adopt a positive attitude towards the use of video games in education. Most teachers think that video games 
could motivate students to learn and playing games can support the development of a variety of skills (Can & 
Cagiltay, 2006; Koh et al., 2011; Razak et al., 2012; Sandford, Ulicsak, Facer, & Rudd, 2006; Wastiau, Kearney, & 
den Berghe, 2009). 

Sandford et al. (2006) conducted a survey to 924 primary and secondary school teachers in England on 
their attitude towards the use of commercial-off-the-shelf (COTS) games. 3 1% of them considered motivating 
students as their reason for using games for instruction. More than 60% of them thought that COTS games improved 
students’ motor/cognitive skills, information and communications technology skills, higher order thinking skills and 
knowledge in a specific area. Wastiau et al. (2009) investigated more than 500 teachers in several European 
countries about their opinions of using games for instruction, and found that most of the teachers have used games in 
their teaching and they wanted to know more about how to use games as teaching tools. They believed that students 
are motivated by DGBL and can obtain certain skill such as social and spatio-temporal skills. Razak et al.’s (2012) 
study revealed that teachers considered challenge, curiosity, pleasure and cooperation as the four main reasons for 
playing computer games for learning. 

However, while the majority of teachers view games as a useful tool in education and are willing to use 
games in the future, they maintain a reserved attitude towards the adoption of games for instruction due to various 
reasons. In Koh et al.’s (2011) study, teachers in Singapore thought games can only be used occasionally. Their 
perception was influenced by external factors such as policies and curriculum, and internal factors such as personal 
interest and attitude towards gaming. Some preservice teachers in Turkey expressed their doubts concerning 
classroom management and educational effectiveness of the computer games on the present Turkish market (Can & 
Cagiltay, 2006). The European teachers were not sure about the positive effects of playing games on students’ 
critical skills and performance in the specific subject matter being taught (Razak et al., 2012; Wastiau et al., 2009). 
Some teachers even considered games a distraction to students (Pastore & Falvo, 2010). The negative views pose 
barriers to the adoption of games in classrooms and also imply the inadequacy of current research concerning DGBL. 

Teachers’ perceptions of using games for education may be also influenced by their own limited experience 
with video games, and others’ perceptions, such as those of students, parents, other teachers, and experts 
(Bourgonjon, Valcke, Soetaert & Schellens, 2010; Selfe & Hawisher, 2004). Adult perceptions of video games in 
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education are influenced by their experiences with video games as entertainment or a reward for hard work or their 
little experiences with video games (Selfe & Hawisher, 2004). While students generally have positive attitude 
toward DGBL, some students do not think games should be used in schools (Sanford et al., 2006), and the negative 
views affect parents and teachers’ views of using games in education (Bourgonjon et al., 2010). 

Clearly, the literature presents the potential for teachers to include game as part of their curricula as well 
barriers teachers face in their adoption of games. When looking at barriers specifically, there are some studies 
examining the barriers to utilization of games in schools. 

Teacher Perceived Barriers to The Use of Video Games in Education 

Based on a review of scholarly papers and reports, we found that the major teacher-perceived barriers to 
implementation of video games in classrooms arise from five aspects: schools, using games, teachers, students and 
theoretical research. 

A number of studies suggest that the current educational framework is a huge barrier to the adoption of 
games in education (Baek, 2008; Egenfeldt-Nielsen, 2004; Koh et al., 2011; Rice 2009; Squire, 2005; Wastiau et al., 
2009). In Koh et al.’s (2011) research, the Singapore teachers highlighted that although the Ministry of Education 
supported the use of games in education, it made no specific policies regarding game adoption, which led to lack of 
instructional game materials, inadequate training for teachers and inadequate administrative support for innovation. 
Other frequently mentioned barriers concerning schools include inflexible curriculum, limited budgets, and lack of 
adequate hardware resources (Baek, 2008; Koh et al., 2011; Rice, 2009). 

Teachers consider games’ weak alignment with curriculum and state standards as a huge barrier (Kirriemuir 
& McFarlane, 2004; Koh et al., 2011; Rice, 2009; Simpson & Stansberry, 2009; Wastiau et al., 2009). The other 
barriers arising from games and using games include inaccurate or inappropriate game content, lack of supporting 
materials, negative effects of gaming, high cost, licensing and technical issues, limited affordances (Baek, 2008; 
Wastiau et al.,2009;) and difficulty in assessing learning from playing games (Razak et al., 2012). 

In respect to barriers relating to teachers themselves, although teachers generally consider their lack of or 
inadequate skills of playing games as a barrier, studies show that this should not be a real barrier (Sandford et al., 
2006; Watson, Mong, & Harris, 2011). Sandford et al. (2006) indicate that meaningful use of game depends far more 
on teachers’ knowledge of the curriculum, their working context and effective use of their existing teaching skills 
than on their skills of playing games. This idea is supported by Watson et al.’s (2011) study. In their study, the 
teacher, who has taught with games for several years, said that a non-gamer could implement a game in his/her 
classroom because the modern students could figure out the gaming problems easily. Some researchers observed that 
teachers lack time to prepare for a lesson using games and also lack the knowledge and skills, professional 
collaboration, and initiatives to explore new methodologies (Kirriemuir & McFarlane, 2004; Simpson & Stansberry, 
2009), though these are not mentioned by the teachers in the studies. 

The other barriers teachers perceive include students’ varied computer literacy, gaming experiences and 
habits (Baek, 2008; Egenfeldt-Nielsen, 2004; Sandford et al., 2006), and little theoretical research on how to use 
games for instruction (Simpson & Stansberry, 2009; Wastiau et al., 2009). 

Method 

This study uses both qualitative and quantitative methods. 15 teachers who have utilized video games in the 
K-12 classroom were asked to brainstorm and record what barriers they and their colleagues have faced in 
implementing games in schools. This was followed up by individual interviews in order to further discuss these 
barriers and their impacts. The written barriers and transcribed interview data were coded in order to identify a list 
of barriers that were utilized to develop a survey which was then sent out to large number of teachers in order to 
identify what teachers perceive are impediments to the adoption and utilization of games in the classroom. 

Step One: Designing a Survey 

Participants. A list of K-12 teachers from local schools in Indiana, US who have utilized video games in 
the classroom for instruction were contacted via a recruitment email. This list was compiled from a previous study 
with IRB approval. Fifteen teachers replied the email and agreed to be interviewed individually either face-to-face or 
through Skype, with the interviews being audiotaped. Of the fifteen teachers, seven were females and eight were 
males. Five of them were teaching in elementary school, and the other ten teachers were teaching in middle school 
or high school or across both. 
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Procedure. Before the interview began, each participant was instructed to list the barriers that inhibited 
them from using video games in formal learning environments. After completing the list, the participant was 
interviewed by one of the authors using semi-structured questions for 20 to 40 minutes. The individual interviews 
focused on obtaining in-depth information about teacher-perceived barriers to the use of the game and teacher 
attitudes toward the use of games. 

After the interviews, the data were transcribed and coded individually by each of the three researchers to 
generate existing barriers facing the use of educational video games in K-12 schools. Then we discussed the 
differences in our barrier codes, and finally reached consensus in the barrier codes generated from the interview data. 
Based on these codes, the three researchers worked together to create two versions of web-based survey-one survey 
for teachers who have used games for instruction, and the other survey for teachers who have not used games for 
instruction. The two versions of the survey had the same 33 statements about barriers to using games, and 
participants of the two populations were asked to rate how much they agreed or disagreed with each statement on a 
Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). The difference between the two versions of 
the survey is that the survey for teachers who have used games for instruction contained questions asking about 
teachers’ experience with using games, while the other version of the survey did not have those questions. 

Step Two: Examining Factor Structure 

Participants. The list of teachers we used to recruit teachers for interviews, along with another list of 
teachers who have not used games for instruction, were contacted via email to fill out the survey on a voluntary 
basis. This list was compiled from a previous study with IRB approval. In total 109 teachers responded, with 28 
male teachers and 81 female teachers, with an age range from 25 to 60. Of the 109 teachers, 57 teachers (16 
elementary teachers, 23 middle/intermediate teachers, 18 senior high teachers) have used games for instruction and 
52 teachers (14 elementary teachers, 15 middle/intermediate teachers, 23 senior high teachers) have not used games. 
Among the teachers who provided information about their years of teaching, 42 teachers has taught for at least 10 
years, 13 teachers has taught ranging from 6 to 10 years, and 10 teachers had less than 5 years of teaching 
experience. The subjects the teachers were teaching included general education (a variety of subjects to the same 
class rather than the specific subject courses), English, science, social studies, mathematics, and other subjects (e.g., 
world language, life skills, engineering). 

Procedure. In our recruitment email to the teachers, we provided the link to an online survey. After we 
obtained the responses from the two populations, we exported the data of each population from Qualtrics to Excel, 
and created a new Excel file to combine the two populations’ responses to the 33 statements for analysis. 

Data Analysis 

Of thel09 teachers who filled out the survey, 5 teachers did not finish it, and thus the responses of 104 
teachers were used for analysis. In order to extract factors from the 104 teachers’ responses to barriers to the use of 
games in classrooms, a principle axes factor analysis was conducted. Both oblimin rotation and varimax rotation 
were run initially, but the results showed the factors were weakly correlated. Therefore, varimax rotation of factors 
was applied, and we used SAS to calculate the regression scores of the factors as factor scores. 

Result 

Result I: Examination of factor structure. Table 1 shows 25 items and their loadings for the 4 interpretable 
factors. Three rounds of analysis using the criterion of eigenvalues greater than 1 were conducted. After the 1 st round 
of data running and analysis, 3 items with factor loadings less than 0.40 across all the factors were removed. After 
the 2 nd round of data running, another 3 items were removed. After running the data in the 3 rd round, 2 items were 
removed based on the same criterion. Finally, an analysis of 25 items using the criterion of eigenvalues greater than 
1 resulted in 3 factors, and the forth factor had eigenvalue of 0.98. An examination of the scree plot of eigenvalues 
showed that the forth factor should be kept. The 4 factors explained 100% of the common variance in the items and 
47.03% of the total variance, and all 25 items loaded above 0.40 on one of the 4 factors. The Cronbach’s alpha 
coefficients for the 4 factors were 0.88, 0.78, 0.81, and 0.70 respectively, which displayed acceptable to high 
reliability. These 4 factors were labeled: challenges of implementing games effectively, challenges with using 
technology, current educational system, and challenges with obtaining games. 
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Table 1. 

Summary of Exploratory’ Factor Analysis Results for Perceptions about Barriers to Using Games in Classrooms 
Using Varimax Rotation (N=104) 

Result II: Analysis of factor scores according to participant gender, teaching level and experience of using 


Factor Loadings 


Item 

F actor 1 

Factor2 

Factor3 

Factor4 

30 Students get sidetracked and do not focus on learning 
when games are used 

0.72 

-0.07 

0.09 

-0.09 

29 It is difficult to manage a gaming class 

0.70 

0.11 

0.1 

0.06 

4 1 Games are unrealistic - there is a disconnect between 
the real world and the game world 

0.68 

0.05 

0.12 

-0.11 

46 It is difficult to evaluate student performance/learning 
from playing video games 

0.65 

0.08 

0.06 

0.09 

40 Lack of clear alignment between games and learning 
objectives 

0.64 

-0.09 

0.39 

-0.05 

37 Games take too long to play through to fit within the 
course structure 

0.58 

-0.13 

0.25 

-0.02 

39 Lack of availability of games that match the teacher's 
subject area 

0.58 

-0.02 

0.28 

-0.13 

42 Educational games are not engaging 

0.56 

-0.05 

0.2 

-0.03 

43 Lack of evidence that games promote learning 

0.53 

-0.22 

0.33 

-0.21 

23 Some games are too complicated for students 

0.52 

0.18 

-0.04 

0.42 

45 Students don’t recognize the value of games for 
learning 

0.50 

0.24 

0.01 

0.02 

27 Teachers lack interest in games 

0.41 

-0.02 

0.15 

0 

21 Inadequate technology support 

0.17 

0.81 

0.12 

0.01 

17 Lack of available technology to play video games on 

-0.14 

0.69 

-0.14 

0.28 

16 Older computer hardware 

-0.13 

0.66 

0.06 

0.36 

18 Lack of administrative support 

-0.05 

0.54 

0.43 

-0.09 

22 Using technology in school presents challenges 

0.1 

0.43 

0.05 

0.18 

33 The pressures of standards-based assessment limit 
creative instruction like games 

0.15 

0.04 

0.6 

0.19 

19 Lack of supporting materials for how to best use the 
game(s) for learning 

0.45 

0.29 

0.6 

0.03 

20 Lack of teacher resources available for training 

0.41 

0.3 

0.52 

0.31 

38 Lack of time for professional development on how to 
use games 

0.46 

0.24 

0.47 

0.31 

34 There is limited or no available time to play games due 
to the need to meet all of the required curriculum 
standards 

0.23 

-0.02 

0.45 

0.05 

15 Price of games is too costly 

-0.18 

0.07 

0.16 

0.66 

14 Lack of available funds for purchasing games 

-0.24 

0.25 

0.13 

0.59 

25 Newer computer hardware does not support older 
game software 

0.13 

0.19 

0.04 

0.59 

Eigenvalues 

6.26 

3.41 

1.11 

0.98 

% of variance 

20.47 

10.3 

8.74 

7.53 


Note: Factor loadings over .40 appear in bold. 
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games for teaching. Table 2 presents a t-test table of the factor scores by participants’ gender. It shows that male 
teachers and female teachers had different opinions about Factor 1, Factor 2, and Factor 4 (t = 2.64, p < 0.01; t = - 
2.82, p< 0.01; t = -2.38, p < 0.05). Figure 2 depicted the means of factor scores for factors 1 to 4 by participants’ 
gender. It showed that male teachers, more than female teachers, regarded Factor 1 as a serious barrier, whereas 
female teachers, more than male teachers, felt Factor 2 and factor 4 were inhibiting them from using games in 
classes. 


Table 2. Paired t-test of factor scores by gender 


Paired-samples 

Group 

N 

Mean 

SD 

t 

Factorl 

Males 

25 

0.42 

0.83 

Z,64 m 


Females 

78 

-0.13 

0.93 


Factor2 

Males 

25 

-0.41 

0.96 

-2.82" 


Females 

78 

0.16 

0.84 


Factor3 

Males 

25 

-0.03 

0.77 

-0.14 


Females 

78 

0 

0.89 


Factor4 

Males 

25 

-0.36 

1 

-2.88' 


Females 

78 

0.09 

0.78 



*p < 0.05. **p < .01. ***p < .001 
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E 

- 0.1 
- 0.2 
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Figure 2. Patterns of means of factor scores by gender 

Table 3 presents a one-way ANOVA table of the factor scores by participants’ teaching level. Teachers of 
different teaching levels viewed Factor 1 and Factor 4 significantly differently, with F (2, 100) = 3.36, p < 0.05, F (2, 
100) = 5.56, p < 0.01. Figure 3 depicts the means of factor scores for factors 1 to 4 by participants’ teaching level. It 
is obvious that primary school teachers considered Factor 1 as a much less serious barrier than middle/intermediate 
and high school teachers did, and senior high school teachers took Factor 4 less seriously than primary and 
middle/intermediate school teachers did. 



Table 3. Analysis of factoi 

r scores by teaching level 





SS 

df 

MS 

F 

P 

Factorl 






Between groups 

5.64 

2 

2.82 

3.36 

0.0386* 

Within groups 

83.8 

100 

0.84 



Total 

89.43 

102 
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Factor2 

Between groups 

1.03 

2 

0.52 

0.63 

0.5325 

Within groups 

81.54 

100 

0.82 



Total 

Factor3 

82.57 

102 




Between groups 

0.24 

2 

0.12 

0.16 

0.8533 

Within groups 

74.41 

100 

0.74 



Total 

Factor4 

74.64 

102 




Between groups 

7.41 

2 

3.71 

5.56 

0.0051** 

Within groups 

66.58 

100 

0.67 



Total 

74 

102 





*p < 0.05. **p < .01. ***p < .001 



Figure 3. Patterns of means of factor scores by teaching level. 


Table 4 displays a one-way ANOVA table of the factor scores by participants’ experience of using games 
for instruction. Teachers who have used games for instruction perceived Factor 1 and Factor 3 significantly 
differently from those who never used games for instruction, with F (l, 102) = 12.65, p < 0.001, 7^ (1, 102) =6.60 ,p 
< 0.05. Figure 4 demonstrates the means of factor scores for factors 1 to 4 by participants’ experience of using 
games for instruction. It is notable that teachers who have used games for teaching viewed Factor 1 and Factor 3 as 
less inhibiting factors than those who never used games for teaching. 

Table 4. Analysis of factor scores by experience of using games for instruction 



55 

df 

MS 

F P 

Factor 1 





Between groups 

9.91 

1 

9.91 

12.65 0.0006*** 

Within groups 

79.92 

102 

0.78 


Total 

89.83 

103 



Factor2 





Between groups 

2.71 

1 

2.71 

3.31 0.0717 

Within groups 

83.33 

102 

0.82 


Total 

86.04 

103 
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Factor3 

Between groups 

4.58 

1 

4.58 

6.60 

0.0116* 

Within groups 

70.71 

102 

0.69 



Total 

Factor4 

75.29 

103 




Between groups 

0.66 

1 

0.66 

0.88 

0. 3503 

Within groups 

75.95 

102 

0.74 



Total 

76.61 

103 





*p < 0.05. **p < .01. ***p < .001 
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Figure 4. Patterns of means of factor scores by experience of using games for instruction 

Discussion 

Four types of barriers to using games for instruction were distracted, and how teacher perceptions are 
influenced by their gender, teaching level and experience of using games were explored. These findings are 
consistent with the results of previous studies (Baek, 2008; Kirriemuir & McFarlane, 2004; Koh et ah, 2011; 
Simpson & Stansberry, 2009; Wastiau et ah, 2009), and complement previous findings. 

Challenges of implementing games effectively include issues about students and teacher interest, student 
attention, class management, game content and its alignment with learning objectives, game availability, and 
assessment. These findings are supported by previous studies (Kirriemuir & McFarlane, 2004; Koh et ah, 2011; 
Simpson & Stansberry, 2009). The result shows that this is a more serious barrier for male teachers than for female 
teachers, which is not revealed in the literature. Perhaps male teachers and female teachers viewed the issues within 
this factor differently. For instance, a high school male science teacher said that even if his students all had their 
assigned laptops, “they still not get many games and install them on their computers. The biggest barrier is probably 
resources, and the second one is finding a good appropriate fit'’. An elementary female teacher said: “I think the 
biggest barrier to using games, especially video games, is that sometimes they’re not educationally focused enough’’. 
Additionally, the survey shows that female teachers, more than male teachers, felt challenges with using technology 
and challenges with obtaining games were inhibiting them from using games in classes. This finding is consistent 
with the result of previous research (Baek, 2008). Many teachers interviewed mentioned inadequate computers and 
technology glitches, and all the teachers interviewed thought lack of funding a big barrier. Some schools do not have 
money to buy any game or games that meet teacher needs, and some schools only have money to purchase very 
basic stuff. Lack of funding for buying games may partially explain why free web-based games and computer games 
attached with the textbook were used most often by the participating teachers who have used games for instruction. 
Therefore, teachers should be provided more technology assistance and financial support for purchasing computers 
and suitable games. 
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Teaching level also affects teacher perceptions of barriers. Elementary school teachers viewed challenges of 
implementing games effectively a less serious barrier than middle/intermediate and high school teachers. Younger 
students usually have lower expectations of game quality than older students and thus are easy to deal with. One 
teacher who taught computers from Grade 1 through8 noted: “My video games with 1 st through 5 th grade work very 
well. 6 th , 7 th , and 8 th , they’re more trained on their home video games, and it’s hard to keep them focus on 
educational video games because the quality of home versions of web-played war games or whatever, is so much 
better than the quality of educational games. You cannot retain, 1 cannot retain their interests.” Furthermore, 
challenges with obtaining games are perceived more serious by primary and middle/intermediate school teachers 
than by senior high school teachers. It is possible that primary and middle/intermediate school teachers use games 
more often than high school teachers due to less academic pressure, so this factor is more outstanding for them. 

The results also indicate that teachers who have used games for teaching viewed challenges of 
implementing games effectively and current educational system as less inhibiting factors than those who have not 
used games for teaching. It is understandable that teachers who have used games at least have overcome the barriers 
at some point and thus are more confident to deal with challenges of implementing games and the existing 
educational system than those who have not used games. 

Conclusion 

This study explores what barriers in using games for instruction US teachers have personally encountered 
or perceived, and how teacher perceptions of barriers are influenced by their gender, teaching level and experience 
of using games for instruction. Due to the small sample size of participants, the study is more of an exploratory 
study. Despite this, it may still serve as a reference for researchers who promote digital game-based learning in 
formal education. Future research could investigate the perceived barriers of larger numbers of US teachers who 
have or have not used games for instruction, and more importantly, their perceptions of how to reduce the barriers. 
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Abstract 

This study aimed at identifying potential differences in behavior, attitudes, and beliefs with respect to 
academic integrity, plagiarism, and the use of anti-plagiarism tools of 96 1 university students from Thailand, 

Taiwan, and the United States. A survey instalment was deployed in three languages at regional universities to both 
undergraduate and graduate student participants. Mean scores on scales were compared by country and level. 

Results showed significant differences among countries as well as among levels. General tendencies showed that 
students in the USA scored more favorably as compared to their peers in the other two countries, controlling for 
level. Differences were also found between levels, with graduates generally scoring higher than undergraduates. 

Introduction 

Increasing trends in higher education toward distance learning are yielding more diverse student 
populations. As a result, institutions experience greater opportunities for cultural exchange; however, they also face 
new challenges to effectively address academic integrity. Plagiarism is especially difficult to control, due to the fact 
that it is intimately linked to research and academic writing skills, and requires higher levels of interpretation along 
with a strong command of language. Studies have found that international students who are writing in a foreign 
language may face additional difficulties to produce academic writing that avoids plagiarism; a lack of confidence 
and general language ability may lead to excessive copying of original sources, especially at a time when an 
abundance of sources is only clicks away (Abasi, Akbari, & Graves, 2006; Hayes & lntrona, 2005; Pecorari, 2006; 
Sun, 2009). Do these students know that they are bending or even breaking the rules of academic integrity? Are 
there any cultural differences in the interpretation of appropriate behavior or the severity of academic misconduct? 

Building on findings from other studies that showed differences in cheating (Chapman & Lupton, 2004) 
and knowledge of correct citing and referencing in academic writing (Wan, Nordin, Halib, & Ghazali, 2011), the 
present study investigates questions about cultural and regional differences in knowledge, attitudes, and behaviors 
related to academic integrity. A comprehensive survey that had previously been developed at Florida State 
University was translated into Thai and Chinese and deployed at local universities in the United States, Thailand, 
and Taiwan. Kim & Wise (2009) had developed a set of questionnaires adopted from previous empirical studies 
(Hayes & lntrona, 2005; Sun, 2009; Wan et al., 2011) to address the original research questions. The survey 
instrument was designed to capture the respondents’ knowledge and awareness about commonly accepted forms of 
plagiarism; attitudes toward plagiarism; behavior; as well as their educational and cultural exposure to the concept 
of plagiarism. Additional questions were included to investigate respondent’s knowledge and attitudes toward the 
use of anti-plagiarism tools. 

The present study extends findings from this survey that were presented in previous years that set the 
baseline for international comparison (Kim & Wise, 2009) and found significant differences between responses from 
American and Taiwanese undergraduate students (Wise & Chen, 2012). The differences found by Wise & Chen 
included knowledge about and behavior related to plagiarism and academic integrity in general, with Taiwanese 
respondents admitting to having copied text portions without proper citation and submitted entire papers they had 
not written. Knowledge about what constitutes plagiarism, as well as attitudes about the severity of specific 
infractions also differed significantly. Interestingly, there were no differences between the respondent’s attitude that 
plagiarism is a serious offense and that it is morally wrong for students to plagiarize. Furthermore, Taiwanese 
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students were not as aware of anti-plagiarism tools in use at their university as American students. The findings of 
that study prompted the researchers to probe further into the topic. 

This year’s iteration of the study included data from Thailand in addition to extended data from the United 
States and Taiwan, including both graduate and undergraduate students. This offered the opportunity to confirm 
previous findings and triangulate the results in order to identify whether differences may co-vary based on region 
(Southeast Asia versus America) or level. Both dimensions are important distinctions that may qualify the previous 
findings and identify more precisely which sub-group may need additional education and feedback on how to write 
academically and how to cite properly. Additionally, one major methodological change was introduced in that the 
survey, which was previously deployed in English only, was translated into both Thai and Chinese. Since language 
skills were identified as a factor that may lead to plagiarism in the context of producing academic texts in a foreign 
language, the researchers wanted to rule out the possibility that language barriers could in any way influence the 
survey responses. With this change, the population of potential participants could be extended to students who are 
not necessarily fluent in English, which had been a limitation of the previous study. 

Research Questions 

flow do students in Taiwan, Thailand, and the USA differ with respect to: 

1 . Their knowledge of what constitutes plagiarism? 

2. Their behavior in terms of academic dishonesty? 

3. Their attitude toward academic honor? 

4. Their awareness of anti-plagiarism tools? 

5. Their beliefs about anti-plagiarism tools? 

6. Their behavior in light of anti-plagiarism tools? 

Method 


Participants 

Participants included both undergraduate and graduate students from universities in Taiwan, Thailand, and 
the USA. Students were recruited for participation via email invitations from professors or department 
representatives at each university. Since participation was entirely voluntary, it was not possible to control the 
sample in terms of gender and level. While the differences were accounted for in data analysis, it should be noted 
that the samples are not necessarily representative of the corresponding populations. Table 1 summarizes the 
distribution of participants by region, level, and gender. 


Table 1 : Participants by Country, Level, and Gender 


Gender by Level 

Taiwan 

Thailand 

USA 

Total 

Graduate Students 

31 

199 

245 

475 

Male 

8 

83 

87 

178 

Female 

23 

115 

149 

287 

Prefer not to answer 

0 

1 

9 

10 

Undergraduate Students 

266 

138 

82 

486 

Male 

101 

40 

40 

181 

Female 

159 

97 

40 

296 

Prefer not to answer 

6 

1 

2 

9 

Total 

297 

337 

327 

961 


Instmment 

The survey instrument was adapted from a survey that was originally developed and validated by Kim and 
Wise (2009) and translated into Thai and Mandarin Chinese. The translations were supervised and verified by 
participating professors from the respective countries. The instmment included of six scales covering participants’ 
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knowledge, behavior and attitudes towards academic integrity, plagiarism, and anti-plagiarism tools (APTs). The 
first section consisted of a set of statements about cases that may constitute plagiarism, such as ‘Plagiarism includes 
paraphrasing source texts without crediting the source’ and participants were asked whether they agree, disagree, or 
were not sure. Items were scored 1 for selecting the correct option, -1 for selecting the incorrect option, and 0 for 
selecting ‘Not Sure’. The Knowledge score was calculated by adding up the score for the six items for a maximum 
score of 6. The second section listed a number of statements that constitute academic integrity violations and 
participants were asked whether they had done them and how many times; for example, ‘Turned in an assignment 
that 1 have partially copied from assignments I previously submitted.’ The Dishonesty score was calculated by 
adding up the score of the individual items for a maximum of 24. The third section consisted of statements about 
academic behavior, such as ‘Cheating is okay if everyone else seems to be cheating,’ and asked the participants to 
indicate their level of agreement with each statement on a five-point scale. Two items were reverse coded. The 
Attitude score was calculated by adding the scores of individual items for a maximum of 60. The fourth section 
included statements about APTs and asked the participants to indicate whether they were aware of them. Three 
items asked whether or not ATPs were experienced, for example, ‘Anti-plagiarism tools are currently available to 
instructors at my university.’ Two items asked how often respondents had experienced the use of APTs, for 
example, ‘Taken a course that used an anti-plagiarism tool.’ The Awareness score was calculated by adding up the 
score on the individual items for a maximum of 13. The fifth section listed a number of beliefs about APTs, such as 
‘No APTs can detect all cases of plagiarism,’ and asked the participants to indicate their level of agreement with 
each statement on a five -point scale. Six items were reverse coded. The Beliefs score was calculated by adding the 
scores of the individual items for a maximum of 50. The sixth section listed a number of behaviors in light of APT 
use, such as ‘If I knew that my work was being checked by an APT, I would try to find out how to avoid committing 
plagiarism,’ and asked the participants to indicate their level of agreement with each statement on a five-point scale. 
Three items were reverse coded. The Behavior score was calculated by adding the scores on the individual items for 
a maximum of 40. A summary of the instrument along with reliability scores is provided in Table 2. 


Table 2: Summary of Instrument with Reliability Estimates 


Section/Dependent Variable 

Scale 

Number 
of Items 

Cronbach’s 

Alpha 

1. 

Knowledge of Plagiarism 

Disagree, Not Sure, Agree (test score) 

6 

N/A 

2. 

Academic Dishonesty Behavior 

Never (0) to Three Times or More (3) 

8 

.823 

3. 

Academic Honor Attitude 

Strongly Disagree (1) - Strongly Agree (5) 

12 

.829 

4. 

APT Awareness 

Never/No (0) , Once/Yes (1) - 5 or more (5) 

5 

.497 

5. 

APT Beliefs 

Strongly Disagree (1) - Strongly Agree (5) 

10 

.772 

6. 

APT Behavior 

Strongly Disagree (1) - Strongly Agree (5) 

8 

.512 


Data Analysis 

The overall mean score differences among countries controlling for level were analyzed using MANCOVA 
with a Bonferroni post-hoc correction. The mean score differences among regions by level were analyzed for each 
variable by using ANOVA with a Bonferroni post-hoc correction. 

Results 

The research questions focused on identifying potential differences in responses based on country with 
respect to the six main variables measured by the survey: knowledge of plagiarism, academic dishonesty behavior, 
academic attitude, APT awareness, APT beliefs, and behavior in response to the use of APTs. 

Summary of Findings 

The data in Table 3 show the mean scores for overall responses compared by country controlling for level. 
Means for scores of respondents from the three countries on all but the sixth variable differed significantly from 
each other. According to these findings, respondents for the USA had a significantly better understanding of 
plagiarism, with a mean score four times greater than respondents from the other two countries. Although the mean 
score between Taiwan and Thailand only differed by .06 the difference was statistically significant due to the large 
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number of participants. US respondents on average self-reported fewer than two incidents of academic dishonesty as 
compared to over eight reported by the respondents of the other countries. While statistically significant, the 
difference between the mean scores of Taiwanese and Thai respondents is only .23. The mean expression of attitude 
toward academic honor was more evenly spread among respondents from the three countries, with Taiwanese 
respondents scoring 43.95 on average, Thai respondents 48.65, and US respondents 51.62. Similarly, although 
means differed significantly among all three groups for APT Awareness, US respondents indicated 1.5 times greater 
awareness than Thai respondents, whose mean scores were 1.1 times greater than those of the Taiwanese 
respondents. APT Beliefs mean scores were more evenly spread and significant across the board, with a 3.29 point 
difference between Taiwan and Thailand, and a 3.74 point difference between Thai and US respondents. Finally, 
none of the differences among the three groups were statistically significant with respect to adjusting their behavior 
in light of APTs being used to check their academic work. 


Table 3: Mean Scores by Country Controlling for Level 


Variable 

Taiwan 

Thailand 

USA 

Knowledge of Plagiarism 

1.19* 

1.13* 

4.94* 

Academic Dishonesty Score 

8.41* 

8.64* 

1.54* 

Academic Flonor Attitude 

43.95* 

48.65* 

51.62* 

APT Awareness Score 

5.44* 

6.18* 

9.32* 

APT Beliefs Score 

30.41* 

33.70* 

37.44* 

APT Behavior Score 

28.84 

29.11 

29.59 


* Mean difference to each of the two other groups is significant at .05 


In the following we inspect each variable separately, differentiating between undergraduate and graduate 
levels within countries. The charts feature two categories for each country in the following order: Taiwanese 
Undergraduates (TWUG), Taiwanese Graduates (TWG), Thai Undergraduates (TLUG), Thai Graduates (TLG), US 
Undergraduates (USUG), and US Graduates (USG). A matrix is included along with each chart marking significant 
differences between group pairs with ‘x’. 

Knowledge of Plagiarism 



Figure 1: Mean Knowledge of Plagiarism Scores for Country by Level 


The chart shows a clear difference between respondents from the USA as compared to the other two 
countries, with no statistically significant difference between US undergraduates and graduates. Thai undergraduates 
showed the lowest knowledge of plagiarism, differing significantly from all other groups, whereas Thai graduate 
students showed no difference to either Taiwanese undergraduate or graduate respondents. Overall, respondents 
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from the two Asian countries scored less than half as compared to US respondents, indicating a significant gap in 
knowledge of plagiarism in these respondents. The items that showed the greatest difference include “Using 
someone else’s idea in your paper without copying any words may not require proper citation,” “Submitting 
someone else’s paper is acceptable if your friend exclusively wrote it for you” and “Completely paraphrasing source 
texts may not require proper citation.” 

Academic Dishonesty 



CountryLevel 

Figure 2: Mean Academic Dishonesty Scores for Country by Level 


According to the chart in Figure 2, the difference between US undergraduate and graduate respondents was 
not statistically significant and the two groups reported between four and five times higher frequencies of academic 
dishonesty behavior. The highest mean score on academic dishonesty was reported by Thai undergraduates with an 
average of 10.4 incidents per respondent. The dishonesty behaviors reported most frequently include “I have copied 
a few sentences from an Internet source without properly citing it,” “I have worked with others on an assignment 
when asked for individual work,” and “I have paraphrased material from a written source without citing it.” 


Academic Honor Attitude 



Figure 3: Mean Academic Honor Attitude Scores for Country by Level 


243 




The chart representing the respondents’ attitude toward academic honor shows that the difference between 
graduates and undergraduates is not significant for any country. In general terms, respondents from Taiwan scored 
lowest and respondents from the USA scored highest on academic honor attitude; however, the differences between 
Taiwanese graduates and Thai undergraduates, and Thai graduates and US undergraduates were not significant. 
Responses differed most significantly for the following items: “I have read my university's academic honor policy” 
and “I am familiar with the content of my university's academic honor policy.” 

APT Awareness 



Figure 4: Mean APT Awareness Scores for Country by Level 


The results for anti-plagiarism awareness are similar to the previous variables in that the US respondents 
are set apart from the other two countries. However, in contrast to the previous variables, there is a significant 
difference between levels of US respondents in that undergraduates reported the use of APTs at a much higher 
frequency than graduates. Nevertheless, the difference between US graduates and all remaining respondents is also 
significant. Asian responses group around a mean between 5 and 6 with the only significant difference being 
between Thai graduates and Taiwanese graduates, who reported the lowest APT awareness. 


APT Beliefs 
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Figure 5: Mean APT Beliefs Scores for Country by Level 

Beliefs about anti-plagiarism tools show a slightly different pattern of response, where US graduates stand 
out as scoring the highest with significant differences to all other groups. Next, US undergraduates, Thai graduates 
and Thai undergraduates set themselves apart with no significant difference among them. The lowest belief score 
was reported by Taiwanese respondents with no significant difference between graduates and undergraduates. The 
greatest differences were observed in the following items: “I am worried that I could get caught by an APT,” “I feel 
offended by the use of APTs because it feels like I am being suspected of plagiarizing,” and “I believe that trust 
between student and instructor is diminished as a result of using APTs.” 

APT Behavior 



Figure 6: Mean APT Behavior Scores for Country by Level 


The last variable measured students’ reported behavior in light of APTs being used to check their academic 
work. Group means ranged from 28 to 3 1 out of a possible 40, representing a moderate positive effect of anti- 
plagiarism tools on students’ academic behavior especially with respect to plagiarism. Significant differences were 
observed between the following groups: Taiwanese undergraduates and US undergraduates, Thai undergraduates 
and Thai graduates, Thai undergraduates and US undergraduates, and US undergraduates and US graduates. Given 
the great differences between levels within the Thai and US respondents, the overall difference among countries 
controlling for level turned out not to be statistically significant for this variable (see Table 3 above). 

Teaching about Plagiarism 

As illustrated in Figure 7, a majority of respondents from Thailand indicated that they were not taught 
about plagiarism until later in their career with 53.3% stating that they learned about plagiarism in college or 
university. This compares to 53.8% of US respondents indicating that they first learned about plagiarism in primary 
or middle school. A majority of Taiwanese respondents (45.5%) also listed early exposure; however, 24.3% of 
Taiwanese respondents selected ‘College or University’ as compared to only 7.6% of US respondents. Finally, 5.4% 
of Thai respondents indicated that plagiarism was never discussed at all as compared to 1.3% of Taiwanese and 
0.3% of US respondents. In a chi-square analysis the differences among countries were significant at the .05 level 
for each category. For Primary or Middle School, Thai respondents differed significantly from the other two; for 
High School, US respondents differed significantly from the other two; for College or University, all three countries 
differed significantly; and for the last category, the difference between US and Taiwanese respondents was 
significant. 
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Figure 7: Teaching about Plagiarism by Country 


Summary 

The findings of this study suggest that there is a significant gap in knowledge about plagiarism between 
respondents from the USA and those of the other two countries. Based on responses to the survey, teaching about 
plagiarism appears to happen later for Thai students and is more evenly spread across school levels for Taiwanese 
students, whereas US students generally tend to learn about plagiarism in earlier grades. The Plagiarism Knowledge 
scores for both Thai and Taiwanese respondents were four times lower than those of US respondents, which clearly 
illustrates this gap. Similarly, the Academic Dishonesty scores were dramatically lower for US respondents as 
compared to those from the other two countries. Could the lack of knowledge about plagiarism be the cause for 
more frequent infractions? This may be a possible conclusion if the infractions were limited to actions that are not 
clearly identified as dishonest behavior. However, the Dishonesty scale included items that were clearly marked as 
inappropriate, for example: “I have worked with others on an assignment when asked for individual work .” “I have 
received unpermitted help from someone on an assignment,” and “I have falsified information on a bibliography.” 
Therefore, the higher dishonesty scores cannot simply be attributed to a lack of understanding. Further insight into 
the difference in behavior is provided by inspecting the respondents’ attitude toward academic integrity. 

While the difference among countries is less pronounced for the Academic Honor Attitude score, the trend 
of US respondents scoring more favorably as compared to respondents from the other two countries is confirmed in 
the data. Interestingly, the items referring to an academic honor policy stood out as particularly differentiating 
between the US and the other respondents. This might indicate that the presence and awareness of such a policy 
impacts students’ behavior and attitudes. Monitoring students’ behavior through anti-plagiarism tools may be done 
in combination with academic integrity policies in order to enhance their effect. When it comes to the awareness of 
and beliefs about APTs, the group differences followed a similar trend as observed for the other variables with 
respect to differences in countries. In fact, with the exception of the last variable that measured respondents’ 
behavior in light of having their work checked by APTs, all the results point to a possible cultural difference 
between Asian and American students with respect to the teaching and practice of plagiarism, which likely 
contributes to student attitudes and beliefs about plagiarism, anti-plagiarism tools, and academic integrity in general. 
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The observation that graduate students generally scored more favorably on the variables of interest as 
compared to their undergraduate peers makes sense based on the premise that students will continue to be exposed 
to concepts of academic integrity over the course of their education. However, it is important to note that the 
differences in level of education were not confounded with the observed differences in country. The findings of this 
study clearly suggest that students in these three countries differ in their knowledge of plagiarism and their attitudes 
toward academic integrity regardless of their level of education. 

Conclusion 

The results have given rise to concern among some faculty who were involved with the study, prompting 
them to consider increasing education efforts to raise awareness of students about plagiarism and how to avoid 
violating integrity expectations in academic work. The importance of producing original work that appropriately 
credits sources is a core value that is being increasingly recognized across the globe especially in light of rising 
international collaboration and technological advances that facilitate sharing and exchange of academic work. It 
should be noted that the study was limited to the participation of individual institutions in each country, and the 
respondents are not necessarily representative of the overall population. Future iterations of this study will aim at 
increasing generalizability by recruiting participants from a greater number of institutions. Furthermore, in addition 
to including data from other countries in future studies, efforts should be made to identify cultural factors that may 
account for specific differences in attitudes, beliefs, and behaviors in order to design appropriate educational 
interventions and set institutional policies to govern and guide the behavior of university students with respect to 
their academic writing. 
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Abstract 

The purpose of this mixed methods study was to examine non-native English speaking students’ activity in 
face-to-face versus online learning environments. This topic is discussed in multiple fields, including issues related 
to the global environment and academic mobility, trends and requirements in education, and intercultural 
communication and languages. This research had the unique opportunity to observe two graduate courses at a major 
U.S. university in the same topic (Statistical Data Management), one online and one face-to-face, taught by the same 
instructor in the same semester. Participants were graduate students and included native and non-native English 
speakers. 

The findings of this study indicate that the activity of non-native English speakers increases in the online 
environment compared to face-to-face education. Nevertheless, non-native English speaking students preferred 
face-to-face courses. This article offers recommendations for those in international higher education to engage 
students actively in English based education independent of their native language. 

Introduction 

International students, many of them non-native English speaking students and scholars, will travel to the 
United States to participate in higher education. Others will prefer to stay in their home countries, or participate 
from varying geographical locations in online programs based in the United States (Giiriiz, 201 1). The Open Doors 
2011 Fast Facts reported a total of 723,277 international students in the United States for the academic year of 2010- 
2011; in the last 10 years, this population has increased by 32% (Open Doors, 2012). Academic mobility refers to 
“scientists and scholars carrying out research and teaching in other countries, and students studying abroad for a full 
degree or as part of their degree requirements back home” (Giiriiz, 2011, pp. 19-20). This growing international 
diversity results in challenges and opportunities for global learning (Gibson, Rimmington, & Landwehr-Brown, 
2008, p. 11). Teaching foreign languages and foreign cultures, as well as international and intercultural 
collaboration, will be a key to successful international and global education (Gibson et ah, 2008; Giiriiz, 2011). 

Online education has become a central pillar in international higher education. The National Science 
Foundation (NSF) defined the concept of online education as “the use of networked computing and communications 
technologies to support learning” (NSF, 2008, p. 5). More than half of the degree-granting postsecondary 
institutions in the United States offered online education by 2001 (National Center for Education Statistics NCES, 
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2003). Ten years later, a survey with a sample of 1,055 colleges and university presidents found that 77% of U.S. 
higher education institutions offered online education in 2011 (Parker, Lenhart, & Moore, 2011). 

In order to achieve high quality (online) learning, the academic lecture, reading, and writing should be 
enhanced with interactive teaching methods, e.g., discussions (Bonwell & Eison, 1991). “Discussion methods are 
among the most valuable tools in the teacher’s repertoire” (McKeachie & Svinicki, 201 1, p. 36). Brookfield and 
Preskill (2005) argued, “virtually everything we know about good face-to-face discussion also applies online” (p. 
220). With today’s technology (e.g., Internet, smart phones), participants may be located in geographically different 
places, while they still participate in the same online course. The challenge of geographical distance and even 
differences in time can be solved with technology (Gibson et ah, 2008). 

Today, English is a well-established language in the global business and academic world. Many scholars 
agree that English has developed as lingua franca in higher education (Bjorkman, 2011; Ferguson, Perez-Llantada & 
Plo, 2011; Kaur, 2010; Matsuda & Friedrich, 2011). “Intercultural communication competence” (Lussier, 2007, p. 
310) is a central element in higher education with a national and international student population. Non-native 
English speaking students and scholars come with a variety of cultural worldviews and expectations. This 
influences the communication in traditional and online educational environments. “Leaders in a globalized world 
need skills that allow them to collaborate, communicate, negotiate, think critically, and gain multiple perspectives 
through dialogic co-construction of meaning with individuals from different cultures” (Gibson et ah, 2008, p. 12). 
Higher education should aim to teach and practice the skills needed in this globalized world. It will be critical to 
integrate and to leverage cultural and language diversity in educational environments. 

First, we look at the global environment and the influence of technology on education. Second, we 
describe the study in more detail before presenting the results. Based on the findings we present recommendations 
and draw conclusions. 


The Global Environment 

The world has become a globalized village with technological advances and Internet technology. “Global 
village[:] A term coined by Marshall McLuhan in the 1960s that refers to a world in which communication 
technology unites people in remote parts of the world” (Martin & Nakayama, 2010, p. 21). Technology has changed 
the way we interact and with whom we communicate. Our environment has become increasingly interconnected. 
According to the statistics of the U.S. Census Bureau (2009), 22% of the jobs in the United States already depend on 
international businesses. For the future, the forecasts predict that this percentage will continue to rise. 

The interconnected environment requires new competencies. Additions and changes of national education 
curriculum become necessary. The American Council on International Intercultural Education (ACHE), in 
cooperation with the Stanley Foundation, identified “fifty-eight global competencies” (The Stanley Foundation, 
1996). These competencies include the “ability to communicate with non-English speaking persons; awareness of 
diversity, similarities, and interdependencies; ability to work in diverse teams; accept responsibility for global 
citizenship; [and] understand that your community may become endangered without global competence” (Stanley 
Foundation, 1996, pp. 36-37). 

Native and non-native speakers need to acquire the language used in the context specific to their academic 
environment (Sibold, 2011). “Academic vocabulary, however, is notably more difficult to learn than conversational 
language because it is more specific and sometimes abstract, making it difficult to grasp” (Sibold, 201 1, p. 24). 
International students need to be prepared to use English in written and spoken form in academic and discipline 
specific contexts (Bjorkman, 2011). Academic language skills influence communication and academic success. 

Technology in Education 

In traditional face-to-face courses, communication is synchronous (at the same time) and in the same 
geographical location, mostly the same classroom. Communication takes place via spoken language. A research 
study of Hlas, Schuh, and Alessi (2008) found an imbalance in spoken face-to-face communication between native 
and non-native English speakers. Non-native English speakers participated less than native speakers in face-to-face 
courses. On the other hand, “the online environment balanced the conversation between native and non-native 
speaker participation both in quantity and quality” (Hlas et al., 2008, p. 364). Online courses can be synchronous, or 
asynchronous (independent of time, but usually within a given timeframe). Asynchronous online discussions are the 
most common teaching method in online or hybrid (face-to-face and online) education. Here, communication 
happens via written text. In asynchronous online courses, participants have the freedom to participate at different 
times in the same course. Students can take their time to compose their contributions. The findings of Hlas et al. 


249 



(2008) supported that the course method (online or face-to-face) has an influence on the quality and quantity of 
communication contributions of non-native English speakers. 

Participants 

This study collected data from graduate students of different nationalities enrolled in one of two courses 
with the same topic (Statistical Data Management), taught by the same instructor. One course was online and the 
other course face-to-face. Nineteen students registered in the online course, and 47 students enrolled in the face-to- 
face course. 

The focus participants of this study were non-native English speakers. However, the researcher invited all 
enrolled students to participate in this study. For this study, a student, living in the United States for more than six 
years, was associated to the native English speaker group. Of the 66 students, 29 native English speakers and all 14 
non-native English speakers agreed to participate in this study. Table 1 shows an overview of the course enrollment 
and study participation. 

Table 1 


Overview of the Course Enrollment and Participation in the Study 


Course Method 

Number of Students 
enrolled 

Non-Native English Speakers 

Native English 
Speakers 

online 

19(4*) 

2 (2*) 

17 (2*) 

face-to-face (F2F) 

47 (39*) 

12(12*) 

35 (27*) 


Note. *Number of students that chose to participate in this study 


All 14 non-native English speakers (online and face-to-face) were students from Asia. The countries of 
origin were Bangladesh, China, India, and Vietnam. In the beginning of this study, the participating non-native 
English speakers had spent one month up to six years in the United States. 

Data Collection and Analysis 

For the study, the researchers conducted interviews and observed interactions between the instructor and 
students, and among the students in both research settings (face-to-face classes and the online environment). 
Observations were made at three course meetings of the face-to-face course, three online discussions, and one 
optional computer laboratory session. Additional data were collected from the university library resources and 
international salary data from the Internet. 

An earlier study of Hlas, Schuh, and Alessi (2008) provided the first research question. The study of Hlas 
et al. focused on the interaction and activity between native and non-native English speaking students. The 
researchers gave permission to use the study design (Hlas et al., e-mail, April 6, 2012). The first question of this 
current study asked, “How does the amount of student discussion vary for students using English as their native 
language versus students using English as a foreign language in traditional (face-to-face) courses versus 
asynchronous online college courses?” (p. 341). 

This study design quantified the number of communication contributions of native and non-native English 
speakers in online and face-to-face courses. The unit of analysis was the single “speech act” (Hlas et al., 2008, p. 
342). A speech act consisted of a number of written or spoken words. The independent variables were speaker-type 
and course-method. A speaker was one student that could be either a speaker-type of native English speaker (NS) or 
a non-native English speaker (NNS). Speaker-type was a dichotomous variable. The variable “course method” 
defined the method chosen to teach the course. This variable had two values: online or face-to-face (F2F). The 
dependent variables were the number of speech acts and the amount of words per speech act. The number of speech 
acts was a continuous variable, counting each verbal or written contribution to the course. The amount of words per 
speech act was continuous and consisted of counting each word of one speech act. T tests analyzed the difference in 
the mean. 

A detailed observation guide supported the data collection in the face-to-face course with up to 50 persons 
in the classroom. Before each observation in the face-to-face course, the researcher passed a numbered seating chart 
around the classroom. Students confirmed their permission to participate by entering their name. If students chose 
not to participate, they added a ‘X’ in their position on the seating chart. Each verbal contribution of the students 
was audio recorded and transcribed. Throughout the course, the researcher kept track of the course contributions of 
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the students by writing down recording time and student number (position in the seating chart). Transcription and 
documentation only distinguished native and non-native English speakers (omitting the names). 

The additional research question in this current study addressed students’ perceptions of advantages (and 
disadvantages) of the two course methods (online and face-to-face courses) for non-native English speaking 
students. 

The lead researcher conducted interviews in person on the university campus. Seven interviews were with 
non-native English speakers. Two of the non-native English speakers were online students. Six interviews were 
with native English speakers enrolled in the face-to-face course. The interviews took about 30 minutes. The 
researcher analyzed and coded the collected data. Based on the codes and memos, themes and categories emerged 
in the qualitative analysis of the data. 


Student’s Activity 

This mixed methods study analyzed the quantity of students’ communication contributions, or speech acts 
(students’ activity). Quantitative data were collected, in the online and face-to face environments. 

Students’ Activities - Online 

In the first online discussion of ‘Statistical Data Management’, the instructions asked each student to post 
one initial introduction contribution, or speech act. The instructions proposed an introduction with about 200-300 
words. The four students, who had agreed to participate in the study, contributed one speech act each. There were 
two native and two non-native English speakers. The average number of words per speech act was lower for the 
non-native English speakers (157 words) compared to the native English speakers (168 words). 

In the second and third mandatory online discussion all students had to write one initial post. Each student 
had to reply to at least two posts of their peer students. The two native English speakers fulfilled the minimum 
number of three posts, overall. For the non-native English speakers, one student posted the minimum requirement, 
and the other student posted one additional optional reply to a peer posting in each online discussion. In the second 
online discussion, the average number of words for each post was 123 words for the two non-native English 
speakers compared to 96 words for the two native English speakers. In the third online discussion, the average 
number of words for each post for the two non-native English speakers was 98 words compared to 126 words for the 
two native English speakers. Table 2 shows an overview of the students’ activity in the online discussions. 


Table 2 

Online Discussions 


Discussions 

Mean 

Native English Speaker 
(n = 2) 

Non-native English Speaker 
(n = 2) 

Introduction 

Speech acts 

1 

1 

(first online discussion) 

Words per speech act 

168 

157 

Second online discussion 

Speech acts 

3 

3.5 


Words per speech act 

96 

123 

Third online discussion 

Speech acts 

3 

3.5 


Words per speech act 

126 

98 


A t test, t( 1 0) = -0.477, p = .644, revealed no significant difference in the number of speech acts in the three 
online discussions between the non-native English speakers (M = 2.67, SD = 1.37) and native English speakers (M = 
2.33, SD = 1.03). A second ? test, t(10) = 0.119, p = .861, revealed no significant difference in the number of words 
per speech act between the non-native English speakers (M= 125.49, SD = 30.28) and native English speakers (M = 
130.28, SD = 58.15). 

In the online environment, native and non-native English speakers produced a similar amount of speech 
acts. There was no significant difference between the number of words per speech act in the online environment. 
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Students’ Activities - Face-to-Face 


The professor usually started the lecture by distributing a handout to each student. In the first class, the 
average number of speech acts per student was higher for non-native English speakers (0.3) compared to native 
English speakers (0.2). The average number of words per speech act was the same (10 words) for all students 
independent of the native language. In the class towards the middle of the semester, the number of speech acts per 
student was lower for non-native English speakers (0.5); also, the average number of words per speech act was 
lower for the non-native English speakers (3 words) compared to native English speakers (10 words). For the class 
towards the end of the semester, the average number of speech acts per students was higher for the non-native 
English speakers (1.3). The average number of words per speech act was lower for non-native English speakers (4 
words) than native English speakers (5 words). Table 3 shows an overview students’ activity in the face-to face 
classes. 


Table 3 

Face-to-Face Classes 


Class 

Mean 

Native English Speaker 

Non-native English Speaker 


Speech acts 

0.2 (n=27) 

0.3 (n=12) 

Beginning of the semester 

Words per speech act 

10 

10 

Middle of the 

Speech acts 

0.7 (n=26) 

0.5 (n=10) 

semester 

Words per speech act 

10 

3 

End of the 

Speech acts 

0.7 (n=20) 

1.3 (n=12) 

semester 

Words per speech act 

5 

4 


In the quantitative analysis, a t test, i(106) = -0.639,/? = .693, comparing the number of speech acts in the 
three face-to-face courses found no significant difference between the non-native English speakers (M= 0.67, SD = 
1.34) and native English speakers (M = 0.51, SD = 1.13). A closer look at the number of words per speech act 
showed the following results: face-to-face course, a t test, t(21.534) = 2.395,/? = .026, comparing the number of 
words per speech act in the three face-to-face courses found a significant difference between the non-native English 
speakers (M= 4.48, SD = 2.82) and native English speakers ( M= 10.04, SD = 8.65). 

Comparing native and non-native English speakers in the face-to-face course, non-native English speakers 
produced a similar amount of speech acts. There was a significant difference between the number of words per 
speech act. The non-native English speakers produced fewer words per speech act than native English speakers in 
the face-to-face course. 

Students’ Activities in Face-to-Face and Online Courses 

The main focus of this study was online and face-to-face activities of non-native English speakers. 
Therefore, the study tested the relationships between non-native English speakers’ activities and course methods. 
The unit of analysis was the speech act. All statistical tests used an alpha of .05 as level of significance. 

An independent t test revealed that there was a significant difference, 1(37) = - 3.357,/? = .002, in the 
number of speech acts for non-native English speakers between the online course ( M= 2.67, SD = 1.37) and the 
face-to-face course (M= 0.67, SD = 1.34). There were more speech acts of non-native English speakers in the 
online course than in the face-to-face course. 

An independent t test found that there was a significant difference, 1(5.065) = -9.757,/? < .001, in the 
number of words per speech act for non-native English speakers between the online course (M= 125.49, SD = 
30.28) and face-to-face course (M= 4.48, SD = 2.82). The amount of words per speech act was higher in the online 
course than in the face-to-face course for non-native English speakers. 

The first research question had asked, “How does the amount of student discussion vary for students using 
English as their native language versus students using English as a foreign language in traditional (face-to-face) 
courses versus asynchronous online college courses?” (Hlas et ah, 2008, p. 341). This research found the following 
results: There were more speech acts and more words per speech act in the online course than the face-to-face 
course. Based on the findings, the online course environment was more favorable for native and also non-native 
English speakers. 
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Advantages of Online versus Face-to-Face Courses for Non-native English-Speaking Students 

The second research question asked if there are advantages in online versus face-to-face courses for non- 
native English speaking students? The researcher analyzed data from course observations, student resources of the 
university, salary data (PayScale, 2013), and individual interviews with native and non-native English speakers. 

In the qualitative analysis, codes and memos built the basis for the categories and themes. Emerging 
themes and categories were as follows: English as a foreign language, course structure, financial commitment to 
education, and communication with the instructor. 

English as a Foreign Language 

All non-native English speakers who participated in the interviews passed the language proficiency 
requirements of the university (TOEFL minimum test score of 79 iBT). However, for some non-native English 
speakers, English was a problem. Students did not understand the instructor and preferred written course material: 
“the slides are very helpful. Sometimes I can’t understand the professor, and I can watch the slide” (NNS, personal 
communication, September 7, 2012). Others could not understand the English spoken in the classroom: “When I 
come here. The first class here in Alabama, they are having kind of Southern accent. A little bit difference for what 
we learn. Standard American English ” (NNS, personal communication, September 27, 2012), and “the only problem 
I had in my first semester was understanding the accent of people speaking English. We are used to speaking 
European English' ' (NNS, personal communication, September 13, 2012). Students that learned English outside the 
USA became accustomed to a different sound (phonetics) of spoken language. An advantage of written language in 
online education, in particular for the non-native English speakers, was that written language does not have 
phonetics, or sounds of language. Accents and dialects were not as prominent in written language as in spoken 
language. Most written academic texts and presentation slides did not favor a regional accent. 

Additionally, written asynchronous communication allowed more time to compose a communication 
contribution or speech act. In the interviews, the researcher asked, “How much time did you take to write an online 
discussion post (or formal E-mail if students had no online experience)?” Native English speakers took on average 
6 to 8 minutes per post or E-mail. Non-native English speakers needed 11 to 14 minutes, almost twice as long. The 
answers supported that non-native English speakers took more time to write a post or formal E-mail. The online 
environment supported the flexible amount of time to answer and contribute to a discussion. 

Non-native English speaking students mentioned that learning would be easy with textbooks in their native 
language (NNS, personal communication, September 7, 2012). Textbooks in foreign languages could accompany 
and increase understanding of the content for non-native English speakers. Based on the information in the 
interviews, a library search with the course title ‘Statistical Data Management’ as the key word produced the 
following results when sorted by language. Table 4 shows the results of a search in the university library online 
resources. The search on January 18, 2013, found 2,486,775 results. 

Table 4 


Library’ Search with the Course Tide ‘Statistical Data Management’ 


Language 

Number of Results 

English 

2, 285,032 

German 

3,008 

French 

1,327 

Spanish 

902 

Portuguese 

533 

Undetermined 

785 


A system-sort by language did not offer any Asian language or Chinese. However, the first seven entries in 
the undetermined language category were in Chinese language from the Database China/Asia on Demand. 
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Course Structure 


According to the data from the interviews, one of the biggest advantages of asynchronous online education 
was the flexibility in the course setup and structure. The flexibility related to time and access to content material 
and lectures. 

The face-to-face course required mandatory physical presence on a specific day and time. In the online 
course, students could adjust learning times closer to their personal schedule instead of a fixed course-schedule. For 
example, one student noted “I will have time, flexi-time ... it is all about, when I have time' 1 (NNS, personal 
communication, September 11, 2012). 

The flexibility of online courses also applied to the time students spent studying the content. Students 
skipped parts of the slides and took more time where needed. A student explained: “I don’t know, 1 just pause the 
video” (NNS, personal communication, September 11, 2012), and “for some content, 1 just read the book. I got the 
sense, and 1 just skipped some of the slides. I think I got it. And sometimes, before the test, I would go back and 
check it. This is really helpful” (NNS, personal communication, September 27, 2012). 

Flowever, this advantage depended on the instructor and the course structure: “For some of the [online] 
classes it will depend how the teacher will set up the classes, as well as the content of the class” (NNS, personal 
communication, September 27, 2012). “If it is a really good organized professor, he or she will put everything 
online, and you can just follow the learning modules and learn it” (NNS, personal communication, October 3, 2012). 

Financial Commitment to Education 

Resident students observed that most international non-native English speaking graduate students expressed 
a great commitment to their education (NS, personal communication, July 25, 2012; NS, personal communication, 
September 27, 2012). The commitment included investment of time and money. 

The estimated cost of attendance at the university for non-resident (international) students was $18,905 for 
one semester (retrieved on January 18, 2013). Students in the course worked towards a Master of Business 
Administration (MBA) or similar degrees. Although incomes in the countries of origin in Asia vary, one semester at 
this university costs about a yearly salary of an employee with a MBA degree in their Asian home country for 
students from Vietnam, Bangladesh, and India (PayScale, 2013). In comparison, in the United States, the costs for 
one semester were only one third of an average yearly income for the comparable population (PayScale, 2013). 

Table 5 shows the average range of yearly pay for an employee with a MBA degree in the USA and the countries of 
origin of the non-native English speakers in the courses. 

Table 5 


Master of Business Administration (MBA) Average Pay per Year 


Country 

* Salary Range for MBA 

USA 

$57,820 -$106,584 

China 

$29,081 - $123,584 

Vietnam 

$12,208 -$53,062 

Bangladesh 

$8,253 - $24,093 

India 

$5,611 -$18,687 


*Note: Salaries (PayScale, 2013) converted to US Dollar on January 28, 2013 
(http://www.fmanzen.net/waehrungsrechner/). 


Some non-native English speakers received their financial support from scholarships and/or their parents 
(NNS, personal communication, September 6, 2012). Other students encumbered themselves with debts (NNS, 
personal communication, November 26, 2012). 

Online education offers the advantage of taking some courses while being in the home country, before or 
after an international experience in the United States. Based on Internet information of the university at the time of 
the study, this option could lower the estimated costs for one semester by $6,505 (for housing, parking, and meals) 
and left only costs for tuition, college fees, and books of $12,400 for one semester. 
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Communication with the Instructor 


Communication with the instructor was a key concern for all students. Most students had the impression 
that the online environment would distance, change, and/or delay the conversations between the instructor and the 
student. One student noted, “I can ask some questions to the teacher, so it will help me [in the face-to-face course]” 
(NNS, personal communication, Sept. 6, 2012). Looking further at the communication theme, students perceived 
the communication with the instructor as influential for the comfort- and confidence-level: “I have the face-to-face 
interaction with the instructor. If 1 have questions, I can ask them ... I have better communication with the 
instructor” (NNS, personal communication, September 27, 2012). Another student noted “I feel insecure if I do not 
see the professor. That means like, I get less help from the professor” (NNS, personal communication, Oct. 3, 

2012 ). 

The unfamiliar distance to the instructor in the online environment made the students feel insecure. 

Students perceived online communication not as equal to face-to-face communication in education. 

Overall Advantages of Online Education for Non-native English Speakers 

The asynchronous online environment has many advantages for non-native English speakers. The written 
language of the online environment solved some problems related to English as a foreign language. There are no 
phonetics or sounds of language. Students could adjust the level of detail and amount of time spent with the course 
content material according to their individual needs and their prior knowledge. Non-native English speakers can use 
a flexible amount of time to write communication contributions, or speech acts (e.g. online discussion posts), in 
asynchronous online education. Based on the opportunity to participate in online education independent of 
geographic location, the financial burden could decline. 

However, non-native English speaking students were skeptical about the online course method: “1 never 
take online courses, before. I am only used to listen to a teacher in a classroom, so I choose face-to-face class” 
(NNS, personal communication. Sept. 7, 2012). 

Discussion and Recommendations 

Higher education in the United States prepares students for job markets around the world. The 
consideration of course communication and language, course design and structure, and teaching methods that 
activate students are essential to utilize the opportunities of a diverse student body in international higher education. 
Use of online technology is a key element in professional international communication, worldwide. Online 
education is one option to become familiar with the course content and improve professional online communication 
skills. Higher education courses should incorporate online education components like online discussions, video- 
conferencing, and professional online communication (European Parliament 2006; Gibson et ah, 2008). 

To aim for the highest possible learning outcome, students also must learn from each other. International 
higher education with native and non-native English speakers should use the opportunity to motivate and activate 
the communication of its students. All students (native and non-native English speakers) can learn from each 
others’ cultures in addition to learning the course content. A non-native English speaker from China observed, 
“there are people from all over the world here in the U.S. We share our culture” (NNS, personal communication, 
September 7, 2012). International higher education can teach content and culture in the broader definition of culture 
by Hall (1981) involving “how people express themselves (including show of emotions)” (p. 16). 

This research built on the study design of Hlas et al. (2008) to quantify students’ activity. The findings and 
observations of this current study strengthen the findings of Hlas et al. In the current study, native and non-native 
English speakers in ‘Statistical Data Management’ were more active in online than face-to-face courses. The 
students produced more communication contributions, or speech acts, and contributed more words per speech act to 
the online course conversations. 

International non-native English speaking students hesitate to try online education; “I never really tried the 
online one, so I cannot tell, and I am afraid to try” (NNS, personal communication, Oct. 8, 2012). In order to 
overcome the hesitant attitude, the introduction to online education may begin in familiar face-to-face course 
settings. Adding mandatory online discussions may increase students’ activity and the number of communication 
contributions, or speech acts. Bailey and Wright (2000) found, “as the review of the literature indicated, the 
majority of the respondents agreed that students who typically did not participate in class tended to participate more 
in online discussions” (p. 10). Native and non-native English speakers can practice academic writing and 
professional online communications with guidance of the instructors. Hybrid courses (combination of face-to-face 
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and online courses) provide a way to keep the comfort of face-to-face communication. Intercultural learning and 
online learning interrelate the two new and challenging issues (Murphy, Gazi, & Cifuentes, 2007). If students were 
familiar with the course content, they became more courageous to consider unfamiliar ways of learning such as 
online education, “if the course is very unfamiliar for me, I would choose face-to-face. If the course will be related 
to what I have learned, I will choose online’’ (NNS, personal communication, September 11, 2012). 

English as a foreign language challenges non-native speakers, especially in the beginning of an 
international education. Dialects and different pronunciations (e.g., accents) as a part of verbal speech of native and 
non-native English speakers causes misunderstanding for native and non-native English speakers (Kaur, 2010). 
Written communication without phonetics (sounds of language) solves some of the pronunciation problems. 

Earlier studies reported that course design and structure were necessary for good online education (Bailey 
& Wright, 2000; Swan, 2002). This current study confirmed the findings. Non-native English speakers connected a 
good structure and full online access of the course material as positive attributes of online education. 

Universities could make an inventory of their library holdings and literature resources based on language. 
Literature in foreign languages can be favorable of students’ learning in international higher education. It may be 
favorable to include options for translations and access to international course material. A good guideline for the 
international literature holdings could be the number of international students speaking a foreign language in relation 
to the fields of study of these students. 

Active learning is one framework that recommends and describes teaching methods to foster activity and 
contributions of students. Depending on the course content, the “Modified Lecture” (Bonwell & Eison, 1991, pp. 7) 
method, where the instructor complements the lecture with two to three minutes of student discussions after 12-18 
minutes of the lecture, are feasible modifications. The short discussions will give students the opportunity to clarify 
questions and process the lecture information (Bonwell, & Eison, 1991). Non-native English speakers may switch 
back to their native language for clarification. All students can engage in short discussions, strengthen 
understanding of content material, and build social and academic communities. Community building can help to 
overcome (international) communication barriers (Murphy et al., 2007). 

Limitations of the Study 

The distinguishing characteristics of the students were native or non-native English speakers. This 
distinction served to gather information about the influence of a student’s non-native English language background 
in an English-based education. A limitation of the study was that it did not specifically look at the cultural 
background of each participant. 

The nature of the courses was also a limitation. Statistics and statistical analysis are very theoretical 
subjects and can create “fear” in students (Bui & Alearo, 2011; Dykeman, 2011). The professional language is 
highly theoretical and mathematical. Both aspects could have limited students’ activity in both research settings 
(online and face-to-face). 

The overall student population within the two courses was 66 students. Only 14 students were identified as 
non-native English speakers. The relatively small number of students was a limitation to generalizing the findings 
accurately to other international educational environments. However, the opportunity to analyze two courses with 
the same course content (Statistical Data Management), taught by the same instructor, in the same semester, in two 
different environments (online and face-to-face), and with native and non-native English speakers was a rare 
research opportunity. 


Considerations for Future Research 

In order to reevaluate and add to the findings of this study, future research should replicate the study and 
gather data from a bigger sample population of native and non-native English speakers in higher education. This 
research built on the study design of Hlas et al. (2008). The current study design further developed the classroom 
observation design with a step-by-step observation procedure. Based on this enhanced design, classroom 
observations with up to 50 persons became feasible. 

This study looked at online and face-to-face courses. Data were collected from two courses with just one 
faculty member. Research comparing implementations of international higher education across different universities 
would be valuable. Data from observations of more courses, possibly across different academic fields, could reveal 
similarities and differences between universities and/or academic fields. The future research should include native 
and non-native English speaking faculty and staff members. 
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Long-term studies can observe non-native English speaking students and scholars in English based higher 
education. Once the students or scholars return to their home countries, future research should track students’ career 
development. What did the non-native English speaker accomplish regarding their career development after one 
year, three to five years, and up to ten years in their home country? Based on the findings of this future research, 
researchers can provide best practice scenarios, tell success stories, and generate theories. 

As a consideration for future research, access to non-native English speakers was a challenge in this study. 
Non-native English speakers appeared to be a group of participants hesitant to participate. In this study, non-native 
English speaking online students only agreed to participate in the study via personal contact of someone within their 
community. Future studies with non-native English speakers should examine multiple ways to access participants 
and encourage participation. 


Conclusions 

The findings indicate that online environments are favorable for non-native English speakers. Although the 
course topic (Statistical Data Management) may be a limitation regarding students’ activity and participation, the 
non-native English speakers became more active in the online course environment and produced more 
communication contributions. The findings support that all students independent of their native language become 
increasingly active in asynchronous online learning environments. Asynchronous online courses are a productive 
addition to traditional face-to-face courses in international higher education. The online environment offers 
advantages especially for non-native English speakers. Nevertheless, non-native English speaking students are 
hesitant to consider online courses. Higher education should find ways to motivate students towards new ways of 
education. 

A globalized approach to higher education with native and non-native English speakers should incorporate 
new ways to communicate. Professional online communication is one element. Today, purposeful usage of 
technology is a key component in international education. Global education should balance content, pedagogy, 
technology, and culture. It will be an advantage in the global world to be able “to deliver culturally sensitive and 
culturally adaptive instructions” (Parrish & Linder- VanBerschot, 2010, p. 1). 

Students and instructors must dare to try unfamiliar methods of education. Instructional designers need to 
be creative and use the opportunities of online education instead of replicating traditional face-to-face learning 
online (The University of Western Australia, 2012). All students and instructors should gain online course 
experience. Online education is not “a savior” (Njenga, & Fourie, 2010). However, without experience it will be 
difficult to make the decision about the best learning options in higher education from an instructor’s or student’s 
perspective. 

This research added a closer look at non-native English speaking students’ activities in online and in face- 
to-face courses. The researchers hope to inspire more ideas for further qualitative and quantitative research about 
English-based (global) higher education. 

For this research, the distinguishing characteristic was native or non-native English speaker. In a future 
global education, the distinguishing characteristic should not be native or non-native speaker of a language. There 
are more characteristics and talents in students other than language. Students come with diverse backgrounds and 
bring different experiences and talents to education. Instructors should emphasize students’ activity and 
collaborative learning to leverage the diverse talents. Many current and future challenges in the world require teams 
with intercultural communications skills, diverse talents, and digital competence. 

There is no “one best way” for such a complex context. However, one aim of this research was to develop 
recommendations for international higher education, and to evaluate advantages of (online) education for non-native 
English speakers. The findings of this study support that active learning and online education in international higher 
education can be beneficial for all students independent of their (native) language. 
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Abstract 

Narrative ability comes before literacy for bilingual students and helps narrow down the gap in text-level 
literacy between English language learners (ELLs) and native English speakers. Kindergarten ELLs are the best age 
group to receive intervention to improve their oral narrative skills. Multimedia stories have potential to assist 
kindergarten ELLs in developing oral narrative skills, but in previous studies, children viewed them without any 
support or interaction with adults, and few studies address how to use multimedia stories as a part of kindergarten 
programs to support oral narrative development of ELLs. Grounded in Kim, Hannafin, and Bryan’s (2007) 
pedagogical framework of technology integration and Plass and Jones’ (2005) model of multimedia-supported 
cognitive processing in second-language acquisition, this paper discusses how to utilize multimedia stories 
effectively in kindergarten programs to support oral narrative development of ELLs. Special focus is placed on how 
to teach story vocabulary words and implement dialogic reading with the support of multimedia stories to improve 
oral narrative skills of ELLs. Finally, future research directions are suggested. 

Keywords: kindergarten English language learners, oral narrative skills, multimedia stories, vocabulary instruction, 
dialogic reading 

English language learners (ELLs) are defined as “students whose first language is not English, and 
encompasses both students who are just beginning to learn English (often referred to as ‘limited English proficiency’ 
or ‘LEP’) and those who have already developed considerable proficiency” (Lacelle-Peterson & Rivera, 1994, p.55). 
They are “not yet developed to the point where they can profit fully from English-only instruction” (August & 
Shanahan, 2006, p.21). Based on a comprehensive review of previous research, the National Literacy Panel on 
Language Minority Children and Youth has identified English oral proficiency as a critical area for language- 
minority children (August & Shanahan, 2006). They find from the literature that language-minority children can 
acquire word-level components (e.g., decoding, spelling) of literacy as well as their monolingual peers, but rarely 
attain the same degree of text-level skills (e.g., reading comprehension). 

Oral English proficiency promotes the development of reading comprehension skills, which consequently 
helps narrow the gap in text-level skills between second language learners and native English speakers (August & 
Shanahan, 2006). Narrative ability facilitates oral language skills (Stadler & Ward, 2005) and precedes literacy for 
bilingual students (August & Shannahan, 2006; Oiler & Pearson, 2002). Given the importance of oral narrative 
skills, it is necessary to monitor and assess ELLs’ oral narrative skills in the early school years, that is, from 
kindergarten to first grade (Uccelli & Paez, 2007). 

Researchers have used various interventions to try to improve young ELLs’ oral narrative skills (Dockrell, 
Stuart, & King, 2010; Uchikoshi, 2005). Reading printed storybooks to ELLs and providing oral summaries can 
enhance their vocabulary acquisition (Collins, 2010) and sentence repetition (Dockrell et ah, 2010), but not their 
narrative skills (Dockrell et ah, 2010). In the studies in which ELLs children watched English cartoon TV programs 
(Uchikoshi, 2005) or children who learn Dutch as a second language viewed multimedia stories (Verhallen, Bus, & 
de Jong, 2006), it is found that both cartoon TV programs with strong narrative components and the multimedia 
features of stories can help improve some aspects of oral narrative skills. 
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Multimedia storybooks (also referred to as CD-ROM storybooks/electronic storybooks) are electronic 
storybooks that “present children’s literature with text and illustrations similar to a traditional text and also include 
elements designed to enhance the reading experience for beginning readers” (Lefever-Davis & Pearman, 2005, p. 
446). The elements may include automatic reading of story, audio effects, graphic animations, written sentences on 
the pages synchronized with the narration, word pronunciation and definition, and games (Eshet-Alkalai & Chajut, 
2007; Lefever-Davis & Pearman, 2005; Pearman & Chang, 2010). Depending on the publishers, different types or 
series of multimedia storybooks vary in their features that support readers. However, multimedia storybooks 
basically have automatic read features, graphic animations and sound effect. Multimedia storybooks excel children’s 
TV shows in availability, variety, temporal flexibility, portability, and interactivity. 

Reading multimedia storybooks has been found to help improve second language learning children’s 
vocabulary and some aspects of oral narrative skills (Eshet-Alkalai & Chajut, 2007; Verhallen et ah, 2006). Eshet- 
Alkalai and Chajut (2007) reveal that reading multimedia stories enabled children who could barely read or speak 
English to master word pronunciation and meaning recognition in the story. Verhallen et al. (2006) find that 
multimedia features of stories were effective for children who learned Dutch as a second language to expand 
vocabulary and syntax and learn implied elements of stories that indicated goals or motives of main characters. In 
the similar form with multimedia stories, children’s cartoon TV programs with strong narrative components better 
fostered kindergarten ELLs’ oral narrative skills than those focusing on phonics and reading fluency (Uchikoshi, 

2005) . Yet, in those studies, only several, not all, aspects of their oral narrative skills got improved, and it is notable 
that children watched multimedia stories or TV shows alone without any support from or interaction with adults. 

The addition of adult support was assumed to generate better effects of using multimedia stories (Verhallen et ah, 

2006) . However, there is little research on how multimedia stories could be used between teachers and ELLs in 
kindergarten settings. 

Dialogic reading is a reading strategy that involves dialogues between an adult and a child during book 
reading and prepares a child to be a story teller (Whitehurst, 1992). During dialogic reading, an adult encourages a 
child’s oral responses and promotes the child’s acquisition of narrative knowledge by using elaborative “wh-” and 
open-ended questions, repeating child’s good answers, modifying child’s utterances, and expanding his/her 
incomplete responses. Use of dialogic reading techniques in narrative conversation between parents and children 
(either ELLs or monolingual English-speaking children) in their home language has been proven to improve the 
children’s oral narrative skills in their home language (Boyce, Innocenti, Roggman, Norman, & Ortiz, 2010; 
Peterson, Jesso, & McCabe, 1999). While there is little research on the effects of dialogic reading in English on ELL 
children’s English oral narrative development, this approach is anecdotally reported to have positive effects on ELL 
children’s emergent literacy (Doyle & Bramwell, 2006). 

Although researchers emphasize that vocabulary supports the development of English narrative skills 
(Dockrell et al., 2010; Lever & Senechal, 2011; Tabors et al., 2001; Uccelli & Paez, 2007; Uchikoshi, 2005), there 
has been little research on how teachers enhance young ELLs’ vocabulary learning supported by multimedia stories. 
Research has reported mixed results as to whether ELLs learn vocabulary automatically by watching multimedia 
stories alone. While some researchers have demonstrated promising results (Eshet-Alkalai & Chajut, 2007; 
Verhallen et al., 2006), others have found that watching TV programs without follow-up reinforcement cannot 
increase ELLs’ vocabulary size (Patterson, 2002) or enable them to learn vocabulary better than those who do not 
watch TV programs (Uchikoshi, 2006). Despite existing literature on how to teach vocabulary words during 
storybook reading or through videos (Lever & Senechal, 2011; Senechal, 1997; Silverman & Hines, 2009), there is 
scarcity of research on how to teach story vocabulary words using multimedia stories. 

Watching children’s TV programs and playing with iPads have become part of many children’s activities at 
home. However, since school and classroom are the major places where ELLs receive English input (Xu, 2010), it 
would be beneficial for ELLs to receive multimedia-enhanced instruction in kindergarten classrooms with teacher’s 
language support. The reality is that there is lack of research on how to use multimedia stories to support vocabulary 
learning and oral narrative development of ELLs in kindergarten settings. 

Thus, the goal of this paper is to propose an approach to implementing vocabulary instruction and dialogic 
reading with the support of multimedia stories to help ELLs improve their oral narrative skills. In the following 
sections, we discuss what oral narrative skills are, in what ways multimedia stories can assist kindergarten ELLs in 
developing oral narrative skills, and how to foster kindergarten ELLs’ vocabulary acquisition and oral narrative 
skills supported by multimedia stories. 
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Narrative Skills 


Labov (1972) defines narrative as “one method of recapitulating past experience by matching a verbal 
sequence of clauses to the sequence of events which (it is inferred) actually occurred” (pp. 359-360). Researchers 
have analyzed children’s narrative skills from different dimensions (Chang, 2004; Lever & Senechal, 2011; Pearson, 
2002; Peterson, Jesso, & McCabe, 1999; Schneider, Dube, & Hayward, 2005; Uchikoshi, 2005; Verhallen et al., 
2006). Those dimensions share some similarities, and the key dimensions of narrative skills can be synthesized as: 
story structure, cohesion knowledge, contextual knowledge, evaluation, storybook language, and syntactic 
complexity (Level & Senechal, 2011; Schneider et al., 2005; Uchikoshi, 2005). 

A well-structured narrative includes setting information, character descriptions, initiating events, responses 
of the characters, plans to solve conflicts, attempts to solve conflicts, consequences, and reactions to events (Labov, 
1972; Stein & Glenn, 1979). Cohesion knowledge shows how well a narrator combines structure, content 
knowledge, and linguistic knowledge, and it can be assessed by the number and variety of connectives used 
(Peterson & McCabe, 1991). In respect to contextual knowledge, Lever and Senechal (2011) point out that 
“decontextualized language such as anaphora can serve as an index of contextual knowledge” (p.4). To be specific, 
it is appropriate to introduce a character by the name or the title, but after that, it is reasonable to mention the 
character by a pronoun or a label. Orientation to context is key to decontextualized language and is significant to 
narrative (Peterson & McCabe, 1994). 

Evaluation is “the means used by the narrator to indicate the point of the narrative (Labov, 1972, p. 366), 
including the reason for telling the story, the meaning of the narrative, or comments about a character, place, thing 
or event (Labov, 1972; Peterson & McCabe, 1983). That is to say, children not only tell what happens in the story, 
but also embed their own evaluation (Labov, 1972; Peterson & McCabe, 1983). Based on previous research, 
Uchikoshi (2005) summarizes seven evaluative devices, including intensifiers, negatives or defeats of expectations, 
references to emotional states or cognitions, reference to physical states, intentions, causal markers, and words with 
high evaluative content. 

Well-read-to children are able to use book language specific to written narrative that was integrated, 
involving, literary and decontextualized when telling a story (Purcell-Gates, 1988, 2001). When well-read-to 
children were asked to tell a story from a wordless book to a doll, the language they used for telling the story’ was 
different from the language they used for telling a personal experience in vocabulary, syntax and degree of 
decontextualization. 

Syntactic complexity can be measured by the mean length of communication in words, number of different 
words, total number of words, and complexity index (Schneider et al., 2005). However, the total number of different 
words is a more discerning measure of English narrative achievement than the total number of words used (Uccelli 
& Paez, 2007). 


Why Use Multimedia Stories for ELLs 

The outstanding strength of multimedia story is that it sets “a mood and context for a story in a highly 
appealing manner” (Lefever-Davis & Pearman, 2005, p.453) and helps activate learners’ story schema and introduce 
story-related vocabulary (Lefever-Davis & Pearman, 2005). These features reduce the cognitive load learners spend 
on decoding and allow them to focus on meaning making instead (Pearman, 2008), and thus are especially useful for 
beginning and struggling language learners (Lefever-Davis & Pearman, 2005; McKenna, 1998). In addition, 
beginning readers need interaction with a variety of texts (Lefever-Davis & Pearman 2005); electronic storybooks 
can be one such resource (McKenna, 1998). Examples of multimedia stories downloadable from App Store are Ugly 
Duckling, Alice in Wonderland, The Princess and the Pea, and Goldilocks and the Three Bears. 

Researchers also point out the downside of using multimedia stories. Multimedia story may make children 
passive receivers of language if they are over reliant on the electronic features to decode words or read the story 
(Lewin, 1996). Also, its embedded games and interactive animations such as flying birds and hidden characters may 
distract learners from the storyline and comprehension (Pearman, 2008; Pearman & Chang, 2010; Trushell, Burrell 
& Maitland, 2001) and negatively affect their stoiy recall (Trushell & Maitland, 2005; Trushell, Maitland & Burrell, 
2003). Arguably, these weaknesses can be overcome by adult interaction with children and their supervision 
(Pearman & Chang, 2010). 

Studies focusing on monolingual children suggest that compared to reading printed storybooks, 
monolingual children who read multimedia stories of the same content make more progress in vocabulary (Korat, 
2009; Korat & Shamir, 2007; Korat & Shamir, 2008), word reading (Korat, 2009; Korat & Blau 2010), reading 
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skills (Pearman, 2008; Pearman & Chang 2010), comprehension (Ciampa, 2012; Doty, Popplewell, & Byers, 2001; 
Pearman, 2008; Pearman & Chang 2010), and overall emergent literacy skills (Shamir, Korat & Barbi 2008). 

Second language learning children also benefit from multimedia stories in vocabulary acquisition and oral 
narrative development (Eshet-Alkalai & Chajut, 2007; Verhallen et ah, 2006). One advantage of multimedia stories 
for ELLs is visual support. Visual support (e.g., photographs, video, objects, diagrams) helps ELLs process language 
and understand concepts (Gottlieb, 2006). Additionally, visual representations may be very important for 
maintaining young children’s attention to the narrative (Gibbons, Anderson, Smith, Field & Fischer, 1986). Sharp, 
Bransford, Goldman, Risko, Kinzer, and Vye (1995) found that dynamic visual support just for the beginning of a 
story helped kindergartners remember the actors and settings, which consequently facilitated their comprehension of 
the short stories they heard. 

Another benefit of multimedia stories for ELLs is that verbal and nonverbal information conveyed 
simultaneously may help them relate what they already know in their first language to the English equivalents. 
Multimedia stories convey meaning through narration synchronized with animation. According to Paivio’s (1986) 
dual coding theory, the language system of human beings operates a verbal system and a nonverbal system 
simultaneously. Activation of either system will activate the other, and both work together to process information 
and generate information that stays in the long-term memory (Paivio, 1986). This is consistent with a study showing 
that prominent auditory and visual features help children extract important information (Calvert, Fluston, Watkins, & 
Wright, 1982). ELLs are usually bilingual students. Bilingual dual coding theory (Paivio, 1986) suggests that the 
verbal system for the first language, as well as for the second language, can be interconnected through the nonverbal 
system. 

Furthermore, multimedia stories make it possible to listen to a word, a sentence, or a story as many times as 
a learner wants. Children’ acquisition of expressive and receptive vocabulary can be enhanced by listening to a 
storybook read multiple times (Senechal, 1997), and repeated exposures to an episode of an educational TV program 
can improve children’s learning and engagement (Crawley, Anderson, Wilder, Williams, & Santomero, 1999). 

How to Use Multimedia Stories in Kindergarten Classroom 


Framework 

To illustrate how to use multimedia stories to support oral narrative development of kindergarten ELLs, we 
use both Kim, Hannafm, and Bryan’s (2007) pedagogical framework for technology integration and Plass and Jones’ 
(2005) model of multimedia-supported cognitive processing in second-language acquisition. In the previous paper 
“What Hinders ESL Teachers in the US integrating Technologies in ESL Classrooms”, we have provided the 
rationale for employing Kim et al.’s (2007) framework to analyze the alignment among the contexts for technology 
integration. Kim et al.’s (2007) framework includes three levels of integration: standards and curriculum reform 
serve as the overarching level (macrocontext); teacher community and professional development are embedded in 
the macrocontext; the innermost level is a technology-supported class (microcontext), which includes student-tool 
interaction, teacher-tool interaction, and teacher-student interaction. We adapt their framework to the ESL context, 
and the adapted framework is shown in Figure 1. 



Figure 1. Framework for technology integration in ESL teaching. This framework is adapted from Kim, 
Hannafm, and Bryan’s (2007) pedagogical framework for technology integration. 
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If Kim et al.’s (2008) framework provides the big picture of technology integration in classrooms, Plass 
and Jones’ (2005) model (shown in Figure 2) offers guidance on how to use multimedia to support students’ second 
language acquisition based on the cognitive processing phases. In their model, multimedia refers to words (spoken 
or written information) and pictures (static or dynamic visual tools). As shown in Figure 2, apperception indicates 
the process of using selected verbal and pictorial information to draw learners’ attention and facilitate their 
comprehension of the incoming information. Comprehension means to organize words and images into verbal 
mental representations and visual mental representations. Following, the verbal and visual mental models of the new 
information need to be integrated, which is known as intake. Finally, learners need to be given opportunities such as 
dialogues to produce comprehensible output in order to develop communicative competency. 



Figure 2. Plass and Jones’ (2005) model of second-language acquisition with multimedia. 

In this paper, we first briefly analyze the macrocontext for using multimedia stories, and then based on 
Plass and Jones’ model, we focus on describing the procedures of implementing vocabulary instruction and dialogic 
reading with the support of multimedia stories in an ESL classroom. While discussion of teacher-student interaction 
is the focus, it will be interwoven with analysis of teacher-tool and student-tool interaction. 

Standards 

Standards guide teachers’ teaching practices, and thus they should be the first thing to check for technology 
integration in kindergarten classroom. It is notable that the Common Core State Standards (CCSS) allow the use of 
various media to present information to kindergarteners, who are expected to confirm understanding of “a text read 
aloud or information presented orally or through other media” (p. 23). With prompting and support, kindergartens 
are supposed to have the capabilities to read stories, ask and answer questions about key details in a story, and retell 
a story with key details (CCSS). While the CCSS target at the general student population, ELLs are expected to meet 
the same high standards with support. The state standards for kindergarteners in English/language arts align with the 
CCSS on kindergarteners’ storytelling capability. 

However, in the TESOL standards, the PreK-K English language proficiency standards for ELLs in the area 
of language arts reveal lower expectations in oral narrative ability of ELLs, and only kindergarten ELLs with high 
English proficiency are expected to produce a narrative. Thus, the intervention that aims at improving oral narrative 
skills would work better for ELL children with middle to high English proficiency level. 

Materials 

Teachers can choose the multimedia stories and download them from the iPad’s App Store. Selection of 
multimedia stories can be based on three criteria: a. they have obvious narrative plots; b. they are not familiar to the 
participating ELLs; c. the language level is appropriate for this age group; d. they are not holiday specific; e. they 
are interesting. If the group is mixed in gender, the story topics can be neutral; otherwise the story topics can be 
either favorable to boys or girls. In respect to story vocabulary words, three to four words per story that are 
“sophisticated and used often in literary and academic texts” (Silverman & Hines, 2009, p.308) can be chosen based 
on the approach used by Beck, McKeown and Kucan (2002). Seven to nine elaborative questions for each story 
would be designed according to Peterson and McCabe’s (2004) criteria: the questions refer to plot, setting, and 
evaluation of characters. That is, they should best relate to the learning of narrative knowledge. 
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Technology-Supported ESL Class 

Teachers can project the story on a large screen when teaching a group of two to four ELL children. By 
learning together, children may feel more comfortable, and can observe their peers’ language use and leam the art of 
conversation turn-taking (Bond & Wasik, 2009). To maximum the opportunities for individual ELLs to speak, the 
group size should be no more than four. If it is one-on-one instruction, the child can operate the multimedia story on 
the iPad by him/herself, but the teacher needs to supervise the student’s operation to minimize the use of embedded 
games or interactive animations irrelevant to the story plot. Since kindergarteners are just beginning to learn to read, 
showing the written sentences to them makes no sense and might actually increase their cognitive load. Without the 
written sentences on the screen, they can focus on listening to the narration and watching the video, which helps 
them to construct meaning. 

According to Plass and Jones (2005), introduction to the learning material is helpful to learners’ 
comprehension of the input. Therefore, teacher can start the storytelling session by briefly introducing the plot of the 
story. Afterwards, teacher can go over vocabulary to facilitate ELLs’ comprehension of the story. Following that, 
ELLs will watch the story for three times- 1 st time without any interruption, the 2 nd time with the task of answering 
questions about the story vocabulary words, and the 3 ld time with the task of answering questions about the story 
plot. 

Apperception-vocabulary’ learning. Teachers can first go over the target words that will appear in the 
multimedia story using visual support, such as cards or pictures, since visual support is important to ELLs’ 
vocabulary acquisition (Gersten & Baker, 2000; Silverman & Hines, 2009). When a group of ELLs learn together, 
they may have different levels of receptive vocabulary, and thus some children may have already known some of the 
words for instruction but other children do not. However, teachers can still teach every word to them either to 
expand their vocabulary or to reinforce their knowledge of the words. 

Comprehension. After ELLs view the story once without any interruption, the teacher can run the story the 
2 nd time and pause at the slide when raising a question about a word. Since the images ELLs initially see from the 
visual support are different from those shown later in multimedia stories, learners can “learn not only the new label 
but to be able to transfer the newly acquired labels to different representations of the referent” (Senechal & Cornell, 
1993, p.365). This procedure of teaching young ELL vocabulary via using videos has proved to narrow the gap in 
vocabulary knowledge between ELLs and non-ELLs (Silverman & Hines, 2009). 

No consensus has been reached on how to teach young ELLs words effectively. Some researchers assume 
that intervener’s expressive vocabulary in narratives automatically affect ELLs’ vocabulary development (Boyce et 
al., 2010), some find that providing rich definitions of target words during storybook reading contribute significantly 
to ELLs’ word learning (Collins, 2010), and still others have found novel ways to introduce new vocabulary (Lever 
& Senechal, 2011; Senechal 1997; Silverman & Hines, 2009). 

The approaches from two studies give us insights on how to teach story vocabulary words with multimedia 
stories. In Senechal’s (1997) study on teaching vocabulary to preschoolers, the intervener pointed to the illustration 
for each target word while reading the narrative, and then paused to ask a “wh” question with the intention to have 
preschoolers speak aloud the word they just heard. For instance, once after the intervener read the target word 
angling , he/she would ask “What is Arthur doing?” (Senechal, 1997, p.130) If preschoolers could not respond with 
the target word, the intervener would ask the question in a more straightforward way: “Can you tell me the word I 
used when I was reading the book?” (Senechal, 1997, p.130) If children still failed to use the target word, the 
intervener would explain the target word; otherwise he/she continued reading the storybook. The results showed that 
preschoolers’ recall and articulation of new vocabulary based on corresponding illustration during the multiple 
readings of storybooks fostered their acquisition of expressive vocabulary. 

While this method of teaching new vocabulary was designed for three- and four-year-old preschoolers, it 
can apply to kindergarteners as well since they are just beginning to learn to read. However, in the approach stated 
above, interspersing questions in the story reading process can be distractive for children to comprehend the story 
and engage themselves. Silverman and Hines (2009) provided good procedures for using videos to teach vocabulary 
to learners from pre-kindergarten through second grade, but they did not specify how to go over target words 
initially, what questions should be raised after video session, and how learners gave examples of words in other 
contexts. Drawing the strengths of both approaches in the two studies, we designed the approach of teaching words 
using multimedia stories stated at the beginning of this section. 

Intake and comprehensible output. During this stage, teachers are supposed to help students integrate the 
verbal and visual information using questions, statements, images, or brainstorming (Plass & Jones, 2005). Thus, 
after ELLs watch the story the 2 nd time, teachers show them the story the 3 rd time and pause at the specific slide 
where the question comes from and raise the question after the narration of the slide. When children watch the story 
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for the 3 rd time, to draw their attention and facilitate their understanding of the story, teachers can moderately use the 
interactive animations that further the story line or reinforce the events. 

Although previous research shows that for second language learning children, viewing multimedia stories 
alone can help them develop oral narrative ability (Uchikoshi, 2005; Verhallen et al., 2006), few studies compare the 
growth trajectories of oral narrative ability of ELLs in multimedia-enhanced instruction with scaffolding versus 
without scaffolding, and it is suggested that scaffolding to second language learning children in their processes of 
using multimedia stories may generate better effects (Verhallen et al., 2006). 

Vygotsky (1962) identifies the zone of proximal development as the gap between a child’s actual 
developmental level and the potential developmental level achievable with adult guidance or collaboration with 
more advanced peers. Based on Vygotsky’s theory. Wood, Bruner and Ross (1976) develop the concept of 
scaffolding and define it as support from teachers, peers, or other resources which enables students to complete tasks 
that they cannot perform by themselves. Literature shows that when teaching ELLs who are developing language, 
teachers should encourage ELLs to participate in conversations and use questions to prompt ELLs to clarify or 
elaborate initial utterances (August & Hakuta, 1997). Additionally, according to Restrepo and Towle-Harmon 
(2008), “Paring children with more proficient students, scaffolding their conversations, and expanding on what they 
say with correct structure or wording are appropriate techniques to build sentence length and complexity” (p. 13). 
Above all, demonstration or modeling is an important scaffolding technique for ELLs, who need to see or hear clear 
examples of what they are expected to imitate (Walqui, 2006). Second language learning children can reproduce the 
linguistic structures that frequently appear in native speaker teacher’s speech, and the rate of reproduction is 
positively related to the frequency of their occurrence in teacher’s speech (Hamayan & Tucker, 1980). 

The techniques of dialogic reading can foster oral narrative development and well satisfy the language 
developmental needs of ELLs. In dialogic reading, the three main techniques to improve oral narrative ability of 
English-speaking children and ELL are asking “Wh-” context-eliciting and open-ended questions (Boyce et al., 

2010; Lever & Senechal, 2011; Stadler & Ward, 2005), modifying children’s utterances (Lever & Senechal, 2011), 
and expanding children’s answers (Boyce et al., 2010; Dockrell et al., 2010; Lever & Senechal, 2011). Research 
shows that parents’ frequent use of “wh-” context-eliciting questions in narrative conversations leads to children’s 
frequent provision of contextual information about when and where in their own narratives (Lever & Senechal 2011; 
Peterson & McCabe, 1994; Peterson, Jesso & McCabe, 1999), which is an indication of decontextualized language. 
Frequent recasting of children’s ambiguous references can improve their contextual knowledge (Lever & Senechal, 
2011). When children use an ambiguous reference that provides no contextual information, the teacher should help 
articulate the reference with a character name or a label (Lever & Senechal, 2011). For instance, when a child says, 
“She is swimming in the river,” the teacher would reword it as “Alice is swimming in the river.” By expanding what 
children have said to what could have been said provides important linguistic features for children to imitate. For 
instance, the intervener can add connectives to children’s incohesive sentences to highlight the importance of 
connectives in a narrative (Lever & Senechal, 2011). 

With respect to teachers’ language use in scaffolding ELLs to improve oral narrative ability, since 
acquisition of new concepts and language is largely based on students’ prior knowledge (Walqui, 2006; Wood et al., 
1976), teachers should try to use language that has already been learned by ELLs and adjust the level of English 
vocabulary and sentence structure according to individual ELLs’ English proficiency level. Considering the number 
of different words used is a measure of English narrative achievement, teachers should use varied words within 
ELLs’ range of knowledge to increase the variety of vocabulary ELLs will use in their own utterances. Sometimes 
teachers may introduce synonyms of the words that ELLs already know. 

Figure 3 shows the procedures of vocabulary instruction and dialogic reading with the multimedia story 
Self Giant as the example. Finally, ELLs should be provided opportunities to tell stories. Depending on ELLs’ oral 
English proficiency, teachers may have ELLs retell stories initially and then proceed to construct narratives for 
wordless picture books or about personal experiences, or have ELLs rotate these two tasks by retelling a story one 
time and producing a narrative another time. While a description of assessment is beyond the scope of this paper, it 
is important to have well-developed instmments and procedures to conduct the assessment. 


Activity 

Questions 

Step 1 : Go over 
vocabulary and 
provide definitions 

• Frighten 

• Bloom 

• Selfish 
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Step 2: Watch the 
story once without 
pause 


Step 3: Vocabulary 
#2 

Watch the story the 
2 nd time, and pause at 
the slide when raising 
a question about a 
word. 

Children are expected 
to speak out the 
word. 

Frighten 

(Pause at the slide with wall and door) 

How did the child feel when the giant shouted? 

Bloom 

(Pause at the slide with trees, a boy and bushes) 
What happened to the flowers after the giant put the 
boy up on a branch of the tree? 

Selfish 

(Pause at the slide with the giant alone) 

What kind of person do you think the giant is? 

Step 4: Watch the 
story the 3 ld time, try 
a few interactive 
activities, and pause 
at the specific slide 
where the question 
comes from and raise 
the question after the 
narration of the slide. 

Why do you think the children liked to play in the garden? 

How did the children feel when the giant shouted? 

What did they do then? 

Why did the giant decide to share his garden? 

What did the giant learn from not sharing his garden? 

What things do you not like to share? 

What happens if you choose not to share? 

Spontaneous questioning: who, what, when where, why and how 
questions. 


Figure 3. Procedures of vocabulary instruction and dialogic reading with an example of multimedia stories 
Self Giant 


Conclusion and Future Research 

A decade ago, Bax (2003) proposed that the ultimate goal of technology integration in teaching English as 
a second/ foreign language is to realize “normalization” of computer-assisted language learning, in which 
“technology becomes invisible, embedded in everyday practice” (p.23). Multimedia stories on iPad, with easy 
access, obvious benefits for ELLs, and the potential to be used at home should have their place in language arts 
education in kindergarten. This paper proposes how to use multimedia stories to foster oral narrative development of 
ELLs in kindergarten programs. It addresses gaps in previous research since few studies have investigated how this 
kind of technology can be used to support ELLs’ oral narrative skills in kindergarten settings. 

In respect to future research, first, it is necessary to explore what modifications are needed to the proposal 
approach. Other empirical studies need to validate the efficacy of the instructional approach proposed in this paper. 
For instance, researchers can verify whether ELL children who watch multimedia stories with the teacher support 
explicated in this paper have better oral narrative skills than those who watch multimedia stories without teacher 
support. More importantly, researchers need to investigate whether ELL children who watch multimedia stories with 
the teacher support proposed in this paper excel those who watch multimedia stories with the teacher support 
provided in conventional story reading activities. 

Additionally, research shows that dialogic reading experience in ELLs’ home language supports their 
narrative development in their home language (Boyce et al., 2010) and also their English acquisition (Huennekens & 
Xu, 2010), and ELLs’ narrative skills in their native language predict their later English narrative quality (Uccelli & 
Paez, 2007). Although most kindergarten programs are English-only programs, how to provide both native language 
support and English support in developing ELLs’ oral narrative ability deserves further exploration. Empirical 
studies would be needed to compare the effects of English-only support, native language-only support, and bilingual 
support on ELLs’ oral narrative development. 
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Abstract 

Research indicates that learning activities are critical to student’s learning, learning satisfaction, and 
perceived learning. To achieve desired learning objectives, students are to be engaged in a variety of learning 
activities in online courses such as interactive learning activities (I), collaborative learning activities (C), authentic 
learning activities (A), reflective learning activities (R). This paper attempts to determine whether there is a 
relationship between the type of learning activities and student engagement in learning (E), learning satisfaction, and 
perceived learning in two online Health Information Administration courses. The results indicate that interacting 
with other students and reflecting on one’s learning process are among the most enjoyable learning activities; 
competing authentic learning activities individually, collaborative learning, interacting with the instructor, reflecting 
on course content are among the enjoyable category; and completing authentic learning activities collaboratively is 
the only one in the lest enjoyable category. The results also indicate that all types of learning activities evaluated in 
the study, except the individual authentic learning activities significantly contribute to students’ satisfaction with the 
course. 


Introduction 

Research indicates that learning activities are critical to student’s learning, learning satisfaction, and 
perceived learning. Students are engaged in a variety of learning activities in online courses such as interactive 
learning activities, collaborative learning activities, authentic learning activities, and reflective learning activities. 
Understanding the effects of different types of learning activities on student’s engagement in learning, perceived 
learning and satisfaction is beneficial to online instructors when designing their online courses. Therefore, it is 
necessary to investigate what types of learning activities are enjoyable and effects on these types of learning 
activities of student’s perceived learning and satisfaction. The findings of the study provide insights regarding the 
usefulness of each type of learning activities evaluated in this study. The results also help online faculty and 
instructional designers better understand students’ perspectives of different learning activities in the online program 
of Health Information Administration. Effective design of these types of learning activities can be incorporated into 
online courses accordingly. 

Interactive Learning Activities 

Research indicates that interaction plays an important role in student learning in both face-to-face and 
online environments. Moore (1989) categorized three types of interaction and labeled them as leaner-content 
interaction, learner-instructor interaction, and learner-learner interaction. Learn-content interaction refers to the 
interaction between the learner and the subject of study, without it learning will not occur. Learning activities in 
online classes need to be designed to promote this type of interaction in a variety of formats to engage learners. 
Learner-learner interaction refers to the interaction between one learner and other learners, alone or in groups, with 
or without the real-time presence of an instructor. This type of interaction is of great importance in education as 
students have to learn skills of group interaction (Phillip, Santoro, and Kuehn, 1988) and be able apply after 
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graduation. Learner-learner interaction can easily be conducted via a variety of communication tools in an online 
class. Learner-instructor interaction refers to the interaction between the instructors and learners, regarded as 
essential by many educators (Moore, 1989). This type of interaction can occur in a number of formats, such as 
instructor responses to postings in a discussion forum, constructive and timely feedback to assignments, responses in 
a brain-storming session. 

Learner-interface interaction was introduced by Hillman, Willis, and Gunawardena (1994). It refers to the 
interaction between learners and the medium in online learning. It is a process of manipulating tools to complete a 
task. They recommended that students be provided with orientations using online technologies to become 
comfortable with technological interfaces. 

Collaborative Learning Activities 

Research indicates that collaborative learning can be effective in learning. Collaborative learning refers to 
learning activities designed for and carried out through pairs or small interactive groups (Barkley, Cross, & Major, 
2005). They described three essential features of collaborative learning: intentional design, co-laboring, and 
meaningful learning. Matthews (1996) stated that collaborative learning is a pedagogy that has at its center 
assumption the students make meaning together and the process enriches their learning. 

In online classes collaborative learning activities can be designed effectively and carried out smoothly with 
appropriate selection of technology. When designing collaborative learning activities key factors such as tasks, 
group composition, distribution of authority, student evaluation, and ease of use of technology need to be taken into 
consideration (Ames, 1992; Barkley, Cross, & Major, 2005; Bean, 1996). 

Authentic Learning Activities 

Learning always occurs in some context, especially in authentic environments (Jonassen, 2011). In 
authentic learning environments, students have opportunities to solve real-world, complex problems via role-playing 
activities, problem-based tasks, and case analysis. Going beyond content, authentic learning involves students in 
immersive environments that requires students using different skill sets and knowledge of multiple disciplines, 
multiple perspectives, ways of working, and habits of mind (Lombardi, 2007). 

Authentic learning provides students opportunities to explore real-world problems, develop and apply their 
problem-solving skills. It is crucial to incorporate authentic learning activities or projects in education, especially in 
advanced classes. This type of learning activities can be carried out in an online environment via simulations, online 
communities, remote instruments etc. It’s critical to design authentic learning activities according to student 
characteristics and provide orientation as necessary. 

Reflective Learning Activities 

Reflection is a process of internally examining and exploring an issue of concern. This process creates and 
clarifies meaning in terms of self and which results in a changed conceptual perspective (Boyd & Fakes, 1983). 
Scanlon and Chernomas (1997) described a three-stage model of reflective learning in nursing education. The three 
stages of reflection include awareness, critical analysis, and new perspective. Students initiate the process of 
reflection in the first stage, awareness. Without this important stage of awareness, reflection will not occur. In the 
second stage, critical analysis, students explore the concept, situation, or event. And in the third stage, new 
perspective, students present the results of the analysis in written format. The new perspective indicates the learners’ 
understanding of the concept, situation, or event. 

Reflective activities fall in two categories, reflections on a student own learning process and reflections on 
the content a student learned or tried to construct their understanding. Reflective learning allows students to think 
more deeply about topics, theories, or concepts that are being explored, resulting in deeper learning. This type of 
activities can be conducted via different tools such as blogs and journals in an online environment. 

The Study 

The study examined the effects of different learning activities in two Health Information Administration online 
courses on student engagement in learning, their satisfaction, and perceived learning and attempted to answer the 
following questions: 

• To what degree do students enjoy each type of the learning activities evaluated in this study? 

• Is there any relationship between the type of learning activities and student engagement in learning, 

satisfaction, and perceived learning? If there is, in what degree? 
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The goal of the first course is to provide the student with a solid working knowledge of medical language 
and basic pharmacology, especially the vocabulary and terminology used in medical coding, classification systems 
and ancillary care. The goal of the second course is to provide an opportunity for students to refine their skills in 
strategic planning in the course of administrative, operational, and project management. Both courses were offered 
in spring 2013 semester and taught by the same instructor who has substantial online teaching experience. 

Instructional designers provided instructional design consultations to the course instructor in the course 
revision of two Health Information Administration online courses in spring 2013 semester. The instructor revised 
the courses by incorporating the iCARE design components. An instrument with 14 questions and four demographic 
items was developed by the research team and was administered in the last two weeks of the semester via 
SurveyMonkey.com. Among the 44 students enrolled in the two classes 29 (n=29) completed the survey with a 
response rate of 66%. The demographics were illustrated in Tables 1. 


Table 1 Student Demographic Summary 



Frequency 

Percentage 

Gender 

Male 

4 

13.8 


Female 

Total 

25 

29 

86.2 

100 

Age Range 

22 to 34 

11 

37.9 

35 to 44 

11 

37.9 

45 to 54 

5 

17.2 

55 to 64 

1 

3.4 

Decline 

1 

3.4 

Total 

18 

100 

Comfortable with 
Technology 

Not very comfortable 

0 

0 

Somewhat comfortable 

0 

0 

Neutral 

1 

3.4 

Comfortable 

14 

48.3 

Very comfortable 

14 

48.3 

Total 

29 

100 

Reason taking this class 

Easy online class 

0 

0 

Convenient online class 

21 

72.4 

Required class 

8 

27.6 

Need 3 credits 

0 

0 

Total 

29 

100 


Results 

Rasch analysis was used to examine the data and answer the first question. The person reliability is 0.78 
and the item reliability is 0.71. The item map (Fig. 1) indicates that interacting with students and reflecting on one’s 
learning process are among the most enjoyable learning activities; competing authentic learning activities 
individually, collaborative learning, interacting with the instructor, reflecting on course content are among the 
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enjoyable category; and completing authentic learning activities collaboratively is the only one in the neutral 
category. 


Figure 1 Item Map of Learning Activities 
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To address the second research question: whether there is any relationship between the type of learning 
activities and student engagement in learning, satisfaction, and perceived learning and if there is, in what degree, 
The Spearman’s rho correlations were computed for the 13 variables with IBM SPSS 21 statistic program. The 
correlation matrix showing the extent of relations among the variables is displayed in Figure 2. 

As illustrated in Figure 2, interaction with the instructor, reflection on own learning process, reflection on 
content learning have strong correlation with students perceived engagement with course content. The correlation 
coefficient between interaction with the instructor and students perceived engagement with course content is 0.518, 
(p=0.004, n=29); The correlation coefficients between reflection on own learning process and students perceived 
engagement with course content is 0.455 ( P — 0 . 1 3. n=29); The correlation coefficients between reflection on content 
and students perceived engagement with course content is 0.541 (p=0.002, n=29). Learning activities involved in 
interaction with the instructor, refection on own learning process, and reflection on course content have mild 
correlations with students perception of engagement in learning with the learning activities with a correlation of 
0.427, ( p=0.021, n=29), 0.442 (p=0.016, n=29) and 0,449 (p=0.014, n=29) respectively. Learning activities 
involved in interaction with other students, collaborative group project, working on real-world projects 
collaboratively have strong correlation with students perception of Engagement in learning by interacting with other 
students, with the following corresponding correlations, 0.583, (p=0.002, n=25), 0.603 (p=0.014, n=16) and 0.639 
(p=0.003, n=19). 

Figure 2 also shows that, all types of learning activities, except the individual authentic learning activities 
significantly contribute to students’ satisfaction with the course. The correlation coefficients between the Interaction 
with other students and students’ satisfaction with the course is r=0.486, (p=0.0 1 4. n=25). The correlation 
coefficients between interaction with the instructor and students’ satisfaction with the course is 0.492 (p=0.007, 
n=29). The correlation coefficient between collaborative learning activity within groups and students’ satisfaction 
with the course is 0.528 (p — 0.036, n=16). The correlation coefficient between authentic collaborative learning 
activities and students’ satisfaction with the course is 0.565 (p — 0.0 12. n=19); the correlation coefficient between 
reflection of learning process and students’ satisfaction with the course is 0.568, (p=0.00 1 . n=29); the correlation 


274 


coefficients between reflection of content learning and students’ satisfaction with the course is 0.539(p=0.003, 
n=29). 

Engaged in learning with the course materials. Engaged in learning with the learning activities and engaged 
in learning by interacting with the instructors are significantly correlated with perceived achievements of the 
learning objectives of the course with the following correlation coefficients reported respectively r=0.479 (p=0.009, 
n=29), r=0.447 (p=0.015, n=29) and r=0.527 (p=0.003, n=29). Interestingly, learning activities involved in working 
on real-world projects collaboratively shows negative correlation with perceived achievements of the learning 
objectives of the course, though there is no statically significance detected. 
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Figure 2 Spearman’s rho Correlations 
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Discussions and Limitations 

The results of the study have several implications. Firstly, learning activities involved in interaction 
between instructor and students play a crucial role in students’ perceived engagement with course content, 
contribute to student’s overall satisfaction of the course and correlate with perceived achievements of the learning 
outcomes. As in face-to-face classroom, students expect that online instructors show their teaching presence by 
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“being there for the students” and “being together with them” throughout their learning experience as well. Garrison, 
Anderson, Rourke, and Archer (2002) defined teaching presence as “the design, facilitation, and direction of 
cognitive and social processes for the purpose of realizing personally meaningful and educational worthwhile 
learning outcomes”. Studies suggest that instructor presence were most important to the students in terms of making 
course requirements clear and being responsive to students’ needs (Sheridan and Kelly, 2010). A rich productive 
online teaching requires that instructor present himself/herself as a real person, encouraging student contributions, 
prompting discussion/interaction, communicating with students about assignment due dates, requirements and 
expectations as well as providing timely, meaningful and constructive feedback on student performance (Sheridan 
and Kelly, 2010, Shea and Bidjerano, 2009). Instructor-student interaction also fosters trust, belonging and support 
and motivates students to better achieve their learning outcomes. 

Secondly, learning activities involved in reflections on content mastering and learning process are essential 
to students’ overall satisfaction of the course and perceived learning. People learn by experiences that allow them to 
absorb the meaning via reading, hearing seeing and feeling, as well as doing the activities and interacting with other 
people (Wertenbroch & Nabeth, 2000). Students also learn by reflection, thinking and linking their recent 
experience to early ones to promote a more complex and interrelated mental schema to develop high order thinking 
skills. The more time the students spend on course content, engage with course materials, and reflect about theories, 
concepts and how and why they get where they are, the better achievements of the learning outcomes they obtain. 
When designing online course activities, instructor may provide ample opportunities for students to reflect their own 
learning process. Hatton and Smith (1995) proposed four activities that can be used to foster reflection: action 
research projects, case and cultural studies, practical experiences and structured curriculum tasks, such as, reading 
fiction and non-fiction, conducting oral interview or writing reflective essays and keep journal. These activities can 
be embedded in course design to enhance students’ overall satisfaction of the course. 

Thirdly, activities related to student-to-student interaction and authentic collaborative group work promote 
the overall satisfaction of the course. Authentic collaborative group activities provide opportunity for students to 
have meaningful online learning experiences of the same richness that his students had previously experienced in 
face-to-face class (Oh and Reeves, 2013). In their research, Oh and Reeves (2013) propose 7 design principles of 
Collaborative group work in an online authentic learning environment. These are 1) facilitate communication, 2) 
establish a strong sense of community and help students have sense of belongingness to their groups and the class, 3) 
provide a variety of technology that everyone can use, 4) maximize opportunities for collaboration and scaffold 
group work process, 5) provide opportunities for establish positive interdependence, 6) Enhance individual 
accountability, motivation, and engagement for active participation in group work and 7) facilitate individual student 
learning about evaluation. These principles provide a framework for faculty to design meaningful and engaging 
collaborative group learning activities to satisfy student’s need. 

While the results support existing literature that learning activities are critical to student’s satisfaction, 
perceived learning, and academic success, this study has its limitations. The sample size is small which limits the 
generalizability of this study. In addition, all participants have the same instructor across the two courses. Students’ 
perception may be affected by this factor. In future studies, these limitations need to be addressed by using large 
sample size and recruit participants from different online course of different subjects. Mixed methods of research 
such as, artifacts, interviews or online student performance observations can also be applied to further investigate 
the relationship between the type of learning activities and student engagement in learning, satisfaction, and 
perceived learning. 
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Abstract 

This study examines the reliability and validity of a Korean version of the Community of Inquiry (Col) 
instrument in online learning. The measurement consists of 34 items to evaluate social presence, teaching presence, 
and cognitive presence and was translated from English into Korean for this study. One Cyber University in Seoul, 
Korea was selected to test the reliability and validity of the Col measurement in Korean. Three factor- structures of 
the Col framework explained 64.67% of the variance in the pattern of relationships among the items. All three 
presences had high reliabilities (all Cronbach’s a > .905). The three-factor structure of the Col framework with 
social presence, teaching presence, and cognitive presence confirms the Korean version of the Col measurement by 
deleting 3 items which cross-loaded on multiple factors. 

Introduction 

Scholars in the distance education field consider the Community of Inquiry (Col) an effective and efficient 
framework of learning within online learning platforms (Akyol & Garrison, 2008; Burgess et al., 2010; Shea & 
Bidjerano, 2009). Based on the Col framework, a varied spectrum of e-learning contexts has been analyzed in order 
to understand implications for online learning practices (Jezegou, 2010; Ke, 2010; Morris, 2011; York, & 
Richardson, 2012). The Col framework reflects the dynamic nature of online inquiry and provides researchers with 
useful guidance when exploring how a sense of community for knowledge construction can be created online and 
significantly impact students’ learning processes (Rovai, 2002; Shea, 2006). The large body of the literature on the 
Col framework focuses generally on online environments in the United States since the Col instrument is in English. 
However, a few research was conducted in various languages regarding the Col measurement although e-learning 
became popular in many different countries. For instance, Moreira, Ferreira, and Almeida (2013) validated the Col 
instmment in Portuguese. The Cronbach’s a of teaching presence, social presence, and cognitive presence in 
Portuguese Col instrument was .93, .89, and .91 respectively. 

E-learning in Korea has been developed rapidly with the wholehearted support from Ministry of Labor of 
Korea (Lee et al., 2009; Lim, 2007). Due to high speed internet and nation-wide broadband infrastructure, high 
quality VOD (Video on Demand) typed e-learning content could be provided for distant learners in Korea 
(Communications Workers of America, 2009; Misko, Choi, Hong, & Lee, 2005). Therefore, the purpose of this 
study is to examine the reliability and validity of the Col instmment in Korean as a starting point of expanding its 
research area to various language and e-learning environment. 

The Community of Inquiry (Col) Framework 

The Community of Inquiry framework consists of three core elements of collaborative constructivist 
learning required to sustain a purposeful learning community: social presence, cognitive presence, and teaching 
presence (Garrison, Anderson, & Archer, 2010). Social presence is defined as the level of recognition of other 
people in the process of communicating with them in online environment (Garrison & Arbaugh, 2007). Social 
presence can be measured by personal ability of developing the interpersonal relations with others through the 
purposed communication process in the online learning community (Swan, Garrison, & Richardson, 2009). 
Cognitive presence originated from the Practical Inquiry (PI) model and focuses on how the learners interact with 
the materials as triggering events, exploration, integration, or resolution (Akyol et al., 2009; Garrison, Anderson, & 
Archer, 2001; Swan, Garrison & Richardson, 2009). Teaching presence is described as thea social and cognitive 
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process of design, facilitation, and direction of the individual learner for get meaningful and educational learning 
outcome (Anderson et al., 2001). The Community of Inquiry (Col) instrument consists of 34 items: 13 items for 
teaching presence, 9 items for social presence, and 12 items for cognitive presence. This instrument has been tested 
to establish a reliable measurement for three presences (Arbaugh, 2007; Arbaugh et ah, 2008) and was validated 
using a multi-institutaional data set (Swan et ah, 2008). The internal consistency reliability of the 34 items s of the 
Col framework was high with Cronbach’s Alpha of 0.91 for social presence, 0.95 for cognitive presence, and 0.94 
for teaching presence (Swan et ah, 2008). 


Methods 


Research Context 

The Cyber University selected for this Study consists of six departments and 19 programs. Over 11,000 
students were enrolled in 2012 and the majority of students were between 19 and 23. All courses in the Cyber 
University are provided through online education and there is no face-to-face component. Most online courses were 
in VOD (Video on Demand) format; however a few were structured as including Problem Based Learning (PBL) 
and discussion based courses. An online survey link to the Col instrument in Korean was posted on the front page of 
the Cyber University Homepage for three weeks from April 15 th , 2013 to May 3 rd , 2013. All survey results were 
collected electronically and coded for analysis. 

Participants 

The participants for this study were 995 Korean undergraduate students: 440 (44.1%) male and 555 (55.9%) 
female students enrolled in the Cyber University. There were 235 freshmen (23.6%), 150 sophomores (15.1%), 325 
juniors (32.7%), and 273 seniora (27.4%). In terms of the locations of residency, the majority of participants (78.2%) 
lived in urban areas such as Seoul, Incheon, and Gyung-gi. 

Data Collection 

Community of Inquiry (Col) Survey. The Col Survey Instrument (Swan et ah, 2008) was administered to 
students to gather data using an online survey (see Appendix A). The 34 Col survey items were measured on a 5 
point Likert scale (Strongly Disagree = 1 and Strongly Agree = 5). The online Col survey in Korean was created by 
using a recognized online survey tool and the students could access to the survey link from the front page of the 
Cyber University Homepage. 

Data Analysis 

Exploratory’ Factor Analysis (EFA) for validity. The exploratory factor analysis was conducted by using 
Statistical Package for the Social Sciences (SPSS, version 21). Exploratory factor analysis is a statistical method to 
increase the reliability of the scale by removing inappropriate items and to identify the dimensionality of constructs 
by examining relations between items and factors when the information of the dimensionality is limited (Netemeyer, 
Bearden, & Sharma, 2003). In this study, the three factors of social, cognitive, teaching presences were used to 
determine the pattern of structure in the 34 item measurement of the Col framework along with a scree plot and 
eigenvalue (Thompson, 2004). The scree test introduced by Cattell (1966) plots eigenvalues against the number of 
factors to determine where a significant drop presents within factor numbers (Netemeyer, Bearden, & Sharma, 2003). 
Kass & Tinley (1979) recommended 5 to 10 participants per item and Comrey & Lee (1992) claimed that a sample 
size 200 is fair and 300 is good. 

Item analysis for reliability. Descriptive statistics were used to evaluate appropriateness of the 34 item 
measurement by calculating the means of all responses and standard deviations (SD) per item. For instance, if a 
mean of an item is close either to 1 or 5, this item may possibly decrease the correlations among the rest of the items. 
Also, the internal consistency reliability was tested by using Cronbach’s alpha for each competency. If the alpha 
value is higher than 0.9, the internal consistency is excellent and if it is at least higher than 0.7, the internal 
consistency will be acceptable (Blunch, 2008). 


Results 


Descriptive Statistics 

Table 1 shows the descriptive statistics including means, standard deviations. Skewness, Kurtosis, 
minimums, and maximums of the three elements of the Community of Inquiry (Col). It reveals that participating 
students perceived high social presence (M=3.44), teaching presence (M=3.88), and cognitive presence (M=3.76). 
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The minimum and maximum are same in three presences as 1 and 5 respectively. In addition, we can assume that the 
data in this study are normally distributed based on the degrees of Skewness and Kurtosis because both are less than 
the absolute value 1 . 


Table 1 . Mean Ratings and Standard Deviations for Each Element of the Community of Inquiry (Col) 



Mean 

Std. Deviation 

Skewness 

Kurtosis 

Min 

Max 

N 

Teaching Presence 

3.88 

.85 

-.553 


1 

5 

995 

Social Presence 

3.44 

.96 

-.175 

-.417 

1 

5 

995 

Cognitive Presence 

3.76 

.82 

-.330 


1 

5 

995 

Total 

3.69 

.87 

-.405 

-.097 

1 

5 

995 


Exploratory Factor Analysis (EFA) for Validity 

An exploratory factor analysis (EFA) was conducted on the 34 items with promax rotation. An initial 
analysis was run to obtain eigenvalues for each factor in the data. The Kaiser-Meyer-Olkin Measure verified the 
sampling adequacy for the analysis, KMO=.978. Bartlett's Test of Sphericity, y2 (561) = 30751.62, p < .000, 
indicated that correlations between items were sufficiently larger for EFA. Three factors had eigenvalues over 1 as 
the scree plot shows clearly in Figure 1 . Additionally, this structure explained 64.67% of the variance in the pattern 
of relationships among the items. The percentages explained by each factor were 55.91% (teaching presence), 5.85% 
(cognitive presence), and 2.90% (social presence) respectively. 


Scree Plot 



Factor Number 

Figure 1. Scree plot for the Korean version of the Community of Inquiry (Col) instrument 

The final three-factor structure in this study is composed of 3 1 items after deleting 3 items which cross- 
loaded on multiple factors. As shown in Table 2, 12 items for factor 1 represent teaching presence, 12 items for 
factor 2 represent cognitive presence, and 7 items for factor 3 represent social presence. The first item that was 
deleted was TP The instructor clearly communicated important course goals because it had factor loading of .785 
on teaching presence and a cross-loading of -.302 on social presence. Then, two items, SP Getting to know other 
course participants gave me a sense of belonging in the course and SP_I was able to form distinct impressions of 
some course participants were deleted because these were loading on the factor different from the intended construct, 
such as teaching presence. 
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Table 2. The Items and Three-Factor Structure of the Community of Inquiry (Col) 



Factor 

1 

2 

3 

TP The instructor helped to keep course participants engaged and participating in 

.885 



productive dialogue. 




TP The instructor was helpful in identifying areas of agreement and disagreement 

.821 



on course topics that helped me to learn. 




TP The instructor was helpful in guiding the class towards understanding course 

.820 



topics in a way that helped me clarify my thinking. 




TP The instructor helped to keep course participants engaged and participating in 

.774 



productive dialogue. 




TP The instructor provided feedback that helped me understand my strengths and 

.768 



weaknesses. 




TP The instructor helped to focus discussion on relevant issues in a way that helped 

.748 



me to learn. 




TP Instructor actions reinforced the development of a sense of community among 

.723 



course participants. 




TP The instructor provided clear instructions on how to participate in course 

.721 



learning activities. 




TP The instructor clearly communicated important course topics. 

.715 



TP The instructor provided feedback in a timely fashion. 

.714 



TP The instructor encouraged course participants to explore new concepts in this 

.695 



course. 




TP The instructor clearly communicated important due dates/time frames for 

.562 



learning activities. 




CP 1 utilized a variety of information sources to explore problems posed in this 


.781 


course. 




CP Learning activities helped me construct explanations/solutions. 


.767 


CP Combining new information helped me answer questions raised in course 


.748 


activities. 




CP 1 felt motivated to explore content related questions. 


.745 


CP Course activities piqued my curiosity. 


.714 


CP I can apply the knowledge created in this course to my work or other non-class 


.711 


related activities. 




CP Problems posed increased my interest in course issues. 


.681 


CP I have developed solutions to course problems that can be applied in practice. 


.679 


CP Brainstorming and finding relevant information helped me resolve content 


.679 


related questions. 




CP Reflection on course content and discussions helped me understand 


.675 


fundamental concepts in this class. 




CP Online discussions were valuable in helping me appreciate different 


.588 


perspectives. 
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Table 2. The Items and Three-Factor Structure of the Community of Inquiry (Col) (continued) 



Factor 

1 

2 

3 

CP I can describe ways to test and apply the knowledge created in this course. 


.548 


SP I felt comfortable interacting with other course participants. 



.925 

SP I felt comfortable participating in the course discussions. 



.786 

SP I felt that my point of view was acknowledged by other course participants. 



.736 

SP I felt comfortable disagreeing with other course participants while still 



.688 

maintaining a sense of trust. 




SP I felt comfortable conversing through the online medium. 



.629 

SP Online discussions help me to develop a sense of collaboration. 



.528 

SP Online or web-based communication is an excellent medium for social 



.477 

interaction. 





Item Analysis for Reliability 

An item analysis was conducted to test the reliability of each element as well as an entire instalment of the 
Community of Inquiry (Col). All three elements had high reliabilities. Cronbach’s a of teaching, social, and 
cognitive presence was .954, .905, and .960 respectively. In addition, Cronbach’s a of the instrument overall 
was .974. 


Table 3. Cronbach s Alpha for Each Element of the Community of Inquiry’ (Col) 



Cronbach’s Alpha 

Cronbach’s Alpha Based on 
Standardized Items 

Number of 
items 

Teaching Presence 

.954 

.955 

12 

Social Presence 

.905 

.906 

7 

Cognitive Presence 

.960 

.961 

12 

Total 

.974 

.974 

31 


Discussions and Conclusions 

The purpose of this study is to test the reliability and the validation of the Community of Inquiry (Col) 
instalment in Korean in an e-learning setting. It was verified that the internal consistency reliabilities of the Col 
instalment in Korean were excellent as a result of item analysis on the items belonged to each presence separately. 
Moreover, this study proved the validation of the Col instalment in Korean with three-factor structures with social 
presence, teaching presence, and cognitive presence which were supported by the literature. 

As a result of descriptive statistics analysis, we could confirm that the data in this study were appropriate to 
conduct exploratory factor analysis (EFA). The data was normally distributed and the 995 sample size was large 
enough for EFA as it was larger than the suggested sample size of 300 (Comrey & Lee, 1992). Based on the results 
of exploratory factor analysis (EFA), this study successfully achieved the simple solution with three-factor structures, 
the same 3 factors as originally proposed by the Community of Inquiry (Col) framework (Garrison Anderson & 
Archer, 2001; 2010) and instalment (Arbaugh et al., 2008; Swan, et. al., 2008) by deleting 3 items which cross- 
loaded on multiple factors. The three-factor structures was previously tested and verified for the English language 
version (Arbaugh, 2007; Arbaugh et al., 2008; Swan et al., 2008). For instance, Arbaugh and colleagues (2008) 
conducted a principal components analysis (PCA) on the 34 items of the Community of Inquiry (Col) instalment 
with 287 graduate students in Education or Business. Their study found that 13 items of teaching presence, 9 items 
of social presence, and 12 items of cognitive presence were significantly loading on factor 1, factor 2, and factor 3 
respectively. The internal consistency reliabilities of each presence were excellent with the Cronbach’s Alpha of 
teaching, social, and cognitive presence at .95, .94, and .91 respectively (Arbaugh et al., 2008). However, for this 
study of the Korean language version, the final three-factor structure is composed of 3 1 items versus the original 34 
of the English language version. Finally, the Korean language version of the Col instalment had final three-factor 
structure composed of 31 items with 12 items for teaching presence, 12 items for cognitive presence, and 7 items for 
social presence. As a result of item analysis in this study, the reliabilities of all three presences were high 
(Cronbach’s a: teaching presence = .957, social presence = .918, and cognitive presence = .960). 
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Two possible causes can be presented for this result. One can be a translation problem. To investigate this 
problem, back-translation method can be adapted. Back-translation method is to re-translate from foreign language 
to original language after translated to foreign language. By comparing between original version and back-translated 
version, the researcher may determine whether the translation caused the problem or not. Another can be cultural or 
environmental difference between the US and Korea. For instance, the major type of delivery methods for online 
education is Video on Demand (VOD) in the participating Cyber University in Korea, whereas reading, discussing, 
and reflecting are main activities in online education in the US. For this reason, when participating Korean students 
read the items for social presence, they might feel lower social presence or higher teaching or cognitive presence 
because participating in online discussion or interacting with other peers are not major components of online courses 
in the participating Cyber University. Therefore, further research is necessary to determine the main cause of the 
cross-loading problems of the Col instrument in Korean. 

The effect of cognitive presence, social presence, and teaching presence from the Col framework on distant 
student’s meaningful learning experience has been verified by the previous literature in the United States. Thus, we 
expect that this study would contribute on expanding the research area of the Col framework to various learning 
environment including different languages and cultures. 
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Abstract 

This study reports that incorporating interactive course elements into a multi-section undergraduate online 
course over a four-year period increased student course completion. We observed that the course pass rate also 
increased over time. We also observed a slight decrease in overall pass rate after additional engagement and support 
features were added. We discuss this finding in light of the curvilinear relationship expected with the application of 
Keller's ARCS-V model of motivational design. This decrease suggests that there is a relationship between the 
course pass rate and the implementation of the ARCS-V model of motivational design within the course. 

Introduction 

Student success is important not only for students, but also for universities. In fact, one measure of the 
quality of university programs often reported is student attrition rates. In a capstone course at a large public 
university, student disengagement and lack of information seeking behaviors were identified as factors that could 
contribute to a high attrition rate. In order for students to successfully complete a capstone course, the students were 
required to be actively engaged in the course and to successfully complete a research project. Therefore, it was 
determined that engagement strategies and scaffolding for information seeking behaviors needed to be implemented 
within the course. These strategies centered on implementing the ARCS-V model of motivational design, thereby 
increasing the attention, relevance, confidence, performance, and volition of the students within the course. 

An ex post facto evaluation of the pass rate, engagement strategies, and information seeking scaffolding 
was completed to determine if there was a correlation between the student pass rate and changes to the motivational 
design included in the course delivery. As expected, when engagement and scaffolding increased, the pass rate also 
increased. Furthermore, an examination of available data also suggested that the pass rate plateaued and even 
decreased slightly as additional engagement features and faculty support were added to the course. This plateau and 
decline were also representative of the curvilinear relationship of the ARCS-V model, and suggested that the pass 
rate was impacted by the motivational design elements of the course. 

Theoretical Framework 

The ARCS model was designed by John Keller with his colleagues at Syracuse University in order to 
consolidate the existing mishmash of motivational theories into a macro-level model and to give a framework for 
applying these concepts to the design of instruction. (Dodge, 2011). This model centers around four tenets: attention, 
relevance, confidence and satisfaction, which form the acronym ARCS. Keller proposed that these tenets must be 
present for sustained learning.. 

Attention is a priori in the ARCS model. Attention may consist of perceptual arousal, inquiry, or variety. 
Prior to any learning, a learner's attention must be focused on the learning task. This focus can be accomplished 
through perceptual arousal. This attention must be maintained in order for the learning to continue. According to 
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Keller, this curiosity can be achieved through creating a sense of mystery, through inquiry arousal — presenting 
problems to be solved through learning — and through varying the instructional presentation (Keller 1987a, 1987b). 

Relevance is the second tenet of the ARCS model. Once a learner's attention is gained, the learner must 
perceive that the learning material is relevant in order to be motivated to learn. Relevance includes the concepts of 
goal orientation, motive matching and familiarity. Through goal orientation, an instructor helps the leaner see the 
usefulness in learning. Motive matching can is useful if the utility of learning material is delayed, as an instructor 
may persuade the learner that the learner will be able to apply the learning material in the future. In addition, the 
way something is taught may also afford relevance. For example, a student may have a vague idea that completing a 
research project is useful. In this type of situation, the learning material can still be made relevant to the learner via 
the instructional methods. Actively engaging a student in self-study helps keep a student motivated. Familiarity, 
which occurs when an instructor helps learners make connections with their experiences, (Driscoll 2005; Keller 
1987a) or when an established relationship exists between a learner and support personnel (Weiler, 2005) is also an 
important part of relevance. 

Confidence is the third element of the ARCS model. Confidence is related to expectancy and attribution 
and includes learning requirements, success opportunities and personal control. When learners believe that they 
have the capacity to be successful, and they are able to identify how to be successful, their learning success is more 
likely. In addition, if students are able to attribute their success to their own efforts, they are more likely to 
experience confidence. (Keller 1987a, 2000). 

The learner must experience some measure of satisfaction from learning. Components of satisfaction 
include natural consequences, positive consequences and equity. The learner must have positive feelings about 
accomplishing the learning tasks. These feelings can be fostered through intrinsic rewards, such as the satisfaction 
that comes with being able to perform a new task, or can be linked to external rewards, such as grades and 
certificates. Satisfaction also can be manifested via the learning process. Learning is a satisfying experience when 
the course design is consistent and appropriate, and the instructor is perceived as fair and impartial (Keller, 2000; 
Driscoll, 2005). It is important to note that an instructor can also contribute to learner dissatisfaction if inappropriate 
strategies are used in instruction. (Keller, 1987a). 

Finally, volition, or self-regulation, is an important factor to consider in motivation. This element was 
added to the original ARCS model to form the ARCS-V model (Keller, 2010). In order for learners to be successful, 
they must maintain goal-oriented behaviors in order to "overcome discouragement and attrition, especially in e- 
learning environments" (Keller, 2008, p. 178). 

In order for motivation to be operationalized, the ARCS categories must be applied to design. In fact, 

Keller has described motivational design as the "marrying a systematic design process with a model of motivation" 
(Dodge, 2011). The ARCS model is not a stand-alone instructional design model. In order to utilize the ARCS 
model, it must be systematically applied to instruction (Keller, 1987c). It is used in conjunction with other models of 
instructional design, such as the Morrison, Ross and Kemp model (Cheng and Yeh, 2009). 

Keller's motivational design system includes twelve steps that may be divided into four categories: define, 
design, develop and pilot. In the definition phase, the audience is analyzed, motivational objectives are established 
and the measures of motivational criteria are defined. (Keller, 1987c). The audience analysis, which may be formal 
or informal, gives the model flexibility. (Dodge, 2011). The results of the audience analysis are reflected on an 
inverted U curve, on which the designer notates which ARCS motivational components are too low, acceptable, or 
too high. This information then is used to establish strategies. 

Potential motivational strategies are identified in the design phase. In this phase, the strategies are also 
selected and a plan for integrating the strategies into the existing instructional design of the course is established. 

The development phase consists of preparing motivational materials, enhancing the existing instructional materials 
and performing a developmental test to discern if the strategies are likely to be effective. Once the developmental 
test is complete, the strategies can then be preliminarily implemented during the pilot phase. The motivational 
strategies are tested with a sample population, their effects are evaluated, and their validity is either certified or the 
strategies are then redesigned before full implementation occurs (Keller, 1987c). 

The overarching concepts of ARCS can be applied to a streamlined approach, which involves a triage 
process to identify motivational challenges and strategies for overcoming these challenges. (Keller, 2000). As the 
designer becomes more familiar with the model and the strategies, its implementation becomes more fluid. (Dodge, 
2011 ). 
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Review of related research 


Attrition is a common problem in distance education. Student attrition can be decreased when student 
engagement is increased (Angelino et al., 2007). This coincides with attention in the ARCS model (Keller, 2008). 
Instructor presence is important to student engagement and perceptions of success in an online course, and this is 
related to attrition rates and student success. It is important that both the instructors and learners have a sense of 
presence. Chickering and Erhmann (1996) proposed that a student is engaged when an instructor exhibits good 
facilitation practices in the online classroom. Online instructors must actively participate. If they do not they are 
perceived as being invisible or absent (Jackson, Jones and Rodriguez, 2010). 

Instructors can be present for students by implementing engagement strategies such as introductions, 
discussion boards, detailed feedback, and email. (Cobb, 2011; Eskey and Schulte, 2012). In addition, instructor- 
biographies can add to a sense of instructor presence (Roblyer & Wienke, 2003), as can a course orientation and 
welcome letter (Lehman & Conceicjao, 2011). Student engagement in online courses also tends to increase student 
success. Student confidence, satisfaction and engagement increase when features such as introductions via 
discussion boards and online services such as library services are embedded in a course. (Angelino et al, 2007; 
Revere & Kovach, 2011). Hence, it can be inferred that when interactive elements that are present in an online 
course, these impact the student successful completion rate. Therefore, when the instructor seeks to engage a 
student by through instructor presence, this impacts the motivation of the student to engage in the course. 

Engagement strategies are crucial for attaining student attention within an online course. Students maintain 
attention when tasks have an appropriate difficulty level (Keller, 2008; Kim & Frick, 2011). It is important for 
instructional topics to be related to student interests and needs. (Weiler, 2005). When topics are related to student 
interest, this increases confidence, relevance and satisfaction (Keller, 2010). In addition, although students have 
been shown to seek help from professors, if the student relies solely on help from a professor or instructor, a student 
will not necessarily succeed in learning. In addition, students will seek help from those with whom they have 
established relationships. (Miller & Murillo, 2011). When a librarian is embedded within an online course, students 
can become partners with both the librarian and the professor. This model has proven to be successful in other 
online capstone courses (Helms & Whitsell, 2013). "The greatest challenge to online education is not the 
technology, but the implementation of strategies and techniques which match the learner with effective learning 
opportunities" (Jackson, Jones, & Rodriguez, 2010, p. 80). Matching students with topics specific to their interests 
and supporting them with an instructor, librarian, and writing support, can contribute to student relevance, 
confidence, satisfaction and volition. 


Capstone course history 

“Department X” (pseudonym) is a department at a large public university in the United States. The 
department offers degrees in liberal studies and professional studies in a variety of concentrations. Prior to 2009, the 
required department capstone project did not have a standardized format. In order to strengthen the credibility and 
rigor of the degree program, the requirements and format of the capstone project were revised in January 2009. 
Creative projects were permitted to fulfill the capstone requirement for some students with degree concentrations 
that would warrant a creative project, such as interior design. But the majority of students were required to submit a 
fifteen-page research paper, exclusive of the title page, abstract, references, and appendices. A text— Booth, Colomb 
and Williams' (2008) The Craft of Research, was selected to provide scaffolding for students. Learners were also 
required to complete timed quizzes about the readings and to submit a proposal, outline, source materials and a 
rough draft at intervals throughout the semester prior to submitting the final draft. In order to graduate, a student was 
required to earn a grade of C-minus or better in the course. (Lattimore, 2009). 

During 2009, the first calendar year these requirements were put into place, the pass rate for the course was 
86%. However in the subsequent two years, the attrition rate increased as pass rates dropped to an average of 84% in 
2010 and 2011. Revisions were made to the course and the pass rate increased to a high of 94% in the Fall of2012. 
However, as improvements continued to be made to the course, the pass rate did not continue to increase, and even 
showed a slight decrease in the Spring 2013 term. 
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Figure 1. Historic pass rate of students enrolled in the capstone course 


Motivational strategies 

In order to reverse the trend in attrition but to maintain the new standards and expectations for graduation, 
the department hired a meta-instructor for the capstone course to oversee and administer the various sections of the 
course. The meta-instructor determined there was a need to provide engagement strategies and scaffolding for 
students. This was accomplished in part by enrolling the students in concentration-specific course sections. Since 
students were required to demonstrate research skills and produce a research paper in order to successfully complete 
the capstone course, it was important to scaffold information seeking behaviors of students enrolled in the course in 
order to give them confidence. This was accomplished by including embedded librarians within the course sections. 
Both the instructors and librarians were selected for the sections based on their expertise in general, and where 
feasible, on specific concentration topics. The instructors were tasked with grading and mentoring the students in 
the course. The librarians were present within the course to answer research questions and scaffold information 
seeking skills. With the implementation of the meta-instructor, topic specific instructors, and embedded librarians, 
the pass rate for the course increased to 87%. 

In the Summer 2012, the meta-instructor implemented a new course shell, online quizzes, an online writing 
lab, and discussion boards. The discussion boards included an "ask the librarian" topic and an introductions topic in 
which the instructors, librarians, and students could introduce themselves. Topics about writing the paper were also 
included. These strategies were implemented to increase the attention, relevance, confidence and satisfaction factors 
within the course. During this term, the pass rate increased to approximately 89%. 

In the Fall 2012 term a teaching assistant who specialized in writing was added to the course to assist with 
writing and grading. The teaching assistant was available to help the students overcome obstacles they experienced 
writing the research paper, thereby addressing the volitional aspect of Keller's ARCS-V model. During this term, the 
mean pass rate increased to 94% with a mode of 100%, a median of 100%, and a range of 29%. 

In the Spring 2013 term, enhanced introductions from the instructors, librarians and teaching assistant were 
added to the announcements section of the course. Video introductions created by the meta-instructor and a "who's 
who" guide was added to the course to help acclimate the students to the course environment. Although the 
additional features were added, the mean pass rate remained at 90% across the sixteen sections of the course. The 
pass rate for this term was multimodal with a median of 87.5% and a range of 28%. 
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Ta ble 1. Engagement features added to the capstone course from January 2009 to May 20 13 

Term Enhancements 

Spring 2012 Meta instructor added, embedded librarians added 

Summer 2012 New course shell, quizzes, online writing lab, faculty and 

student introductions, ask the librarian discussion topic 
Fall 2012 Teaching assistant added 

Spring 2013 Enhanced faculty introductions from instructors, 

teaching assistant and librarians; a who's who guide 
to support personnel, enhanced grading guidelines, 

video introductions 

Although the pass rate of the course decreased between these two semesters, the level of engagement strategies 
increased. In addition, neither the GPA for students enrolled in the course nor the Student Evaluation of Teaching 
(SETE) scores of the section instructors varied significantly across semesters. In the Fall 2012 term, the mean GPA 
across the sections was 3.17, the median was 3.16, the range was 1.18 and the standard deviation was 0.1. During 
this term the GPA was multimodal. In the Spring 2013 term, the mean was 3.056, the median was 3.09, the mode 
was 2.57, the range was 1.2, and the standard deviation was 0.18. 

T abl e 2. Comparison of GPA data between the Fall 2012 and Spring 2013 t erms 


GPA data 

Fall 2012 
(94% |i pass rate ) 

Spring 2013 
(90% p. pass rate) 


3.17 

3.056 

median 

3.16 

3.09 

mode 

multimodal 

2.57 

range 

1.18 

1.2 

c 

0.1 

0.18 


Additionally, there was not an apparent instructor quality difference based on SETE scores. In the Fall 2012, the 
mean instructor SETE score was 4.47, the median was 4.67, the mode was multimodal, the range was 1.75, and the 
standard deviation was .22. In the Spring 2013 term, the mean instructor SETE score was 4.44, the mean was 4.54, 
the mode was 5, the range was 2.67, and the standard deviation was 0.4. 

Table 3. Comparison of SETE data between the Fall 2012 and Spring 2013 terms 

SETE data 

Fall 2012 
(94% |i pass rate ) 

Spring 2013 
(90% |i pass rate) 


4.47 

4.44 

median 

4.67 

4.54 

mode 

multimodal 

5 

range 

1.75 

2.67 

a 

0.22 

0.40 


There is not a significant difference in the overall GPA of the students. The mean SETE score of instructors does not 
show a significant difference between terms, at 4.44 for the SPRING 2013 term and 4.47 for the Fall 2012 term. In 
fact, the SETE mode for the Spring 2013 term was 5, which was higher than the mode for the Fall 2012 term, which 
was 4.47. 

Because of the decrease in pass rate between the Fall 2012 and Spring 2013 terms, the increase in 
engagement features, and the static GPA and SETE ratings, we suggest that the curvilinear relationship of the ARCS 
model is evident within the course, and is reflected in the course pass rate. As Keller has suggested, tension must 
exist in order to promote motivation, but too much tension can be counterproductive (Keller, 2010). 

Limitations and Recommendation 

Although the slight decrease in the course pass rate in the Spring 2013 term suggests a curvilinear pass rate, 
a decrease in the pass rate has only been documented for one term. Furthermore, this study is limited to one multi- 
section course at one university. The relationship that was examined is ex post facto, and no qualitative data was 
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included. A next step could be that some of the motivational strategies implemented in the Spring 2013 term be 
removed from the course and that the pass rate continue to be examined in order to determine if this strategy 
reduction impacts the pass rate. Student perception of the strategies could also be documented in order to triangulate 
these findings. 


Conclusion 

Tension must be present in the motivational design of any course. (Keller, 2010). In a multi-section online 
course, strategies were implemented in order to increase motivation. However, just as Keller's ARCS model of 
motivational design is curvilinear (Keller, 1987a), the pass rate of online courses to which the model is applied is 
also suggested to be curvilinear. This study suggests that through carefully monitoring motivational strategies and 
the pass rate of students enrolled in a course, an optimal pass rate could be reached. We suggest that motivational 
strategies and the pass rate of online courses continue to be investigated as an area of further research. 
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Abstract 

The use of social media has become pervasive, and it has been impacting the way people learn and interact 
with each other in general and their education in particular. Five faculty members at a higher education institution 
were interviewed to examine how they use social media in their courses. The interviews were followed by a student 
survey targeting students in the respective courses. Results revealed that faculty and students perceived social media 
as playing an important role in (a) extending learning beyond the classroom, (b) triggering students to use new 
technology, (c) giving the students an opportunity to reflect on their learning, and most importantly (d) facilitating 
and enhancing students’ learning. 


Introduction 

The rapid technological change that we have witnessed in the last ten years has led to a wired universe 
which has impacted the way people interact with each other as well as the way they process the wealth of 
information that they are surrounded with. The world is even experiencing a “Global Living Room” where people 
interact with each other on social media even when they are watching television (Bannon, 2012). Not only has social 
media influenced individuals on the personal level, but they also have impacted work environments and learning 
environments. Learning is no longer limited to brick-and-mortar settings, rather, it has become ubiquitous where it 
can happen anywhere and anytime due to the abundance of cloud-based technologies that can be easily used and 
accessed (Johnson, Adams, & Cummins, 2012). This has brought to surface inquisitions on how people benefit from 
the ubiquitous information and interaction and how it might impact their knowledge and education. Many research 
studies have described the different use of social media in education (Heibergert, & Loken, 2011; Cain & Policastri, 
2011; Kelm, 2011), and some studies have suggested the positive impact of social media on students’ learning 
(Oskoz & Elola, 2011; Yang & Chen, 2012; Churchill, 2009) , however, little research has looked at how faculty 
members are using social media across the curriculum and how faculty perceive the role social media plays in 
student learning compared to their students’ perceptions. Understanding this role and the learning activities that are 
assigned through social media can inform faculty members in implementing social media in their courses. 

Social Media 

Social media, also known as social software or Web 2.0 tools, refers to a group of technology tools that 
promote a personal repertoire of information through interaction and communication in an online, internet-based 
application (Anderson, 2004; Dabbagh & Reo, 2011; Cistek-Chandler, 2012; Chen and Bryer, 2012). Social media 
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emphasizes user-generated content and sharing publically. Characteristics of Web 2.0 allow more user control over 
technology to create, publish, and co-create which give the user the opportunity to write on the web and produce 
rather than only read, which is the main feature of Web 1.0 technologies (Dabbagh & Reo, 2011). 

Mason and Rennie (2010) emphasize the construct of collective intelligence that is harnessed by social 
media networking through the collaborative efforts of the users while Bates (2011) presents the term “E-learning 
2.0” as suggested by Downes (2005) to refer to social media’s pedagogical use in education. Bates elaborates on 
Web 2.0 affordances and states “that they empower the end user to access, create, disseminate and share information 
easily in a user-friendly open environment” (p. 25). For the purpose of this study, I refer to social media as tools that 
“foster and encourage informal conversation, dialogue, collaborative content generation, and the sharing of 
information” (McLoughlin & Lee, 2011, p. 45) and serve as means to facilitate and enhance student learning. 

Social media tools are used for different purposes although they share a common process and outcome by 
affording interactivity, communication, and sometimes creativity between the end users. While some of these tools 
promote collaboration between users to produce some kind of a collaboratively created work (Wikis, DropBox, 
Google Docs), others allow for connecting and socializing with friends around the world (Facebook, Friendster, 
Google +). Other social media tools support media sharing (YouTube, Flickr) while some allow bookmarking 
(Delicious, Diigo) (Dabbagh & Reo, 2011; McLoughlin and Lee, 2011; Bates, 2011). 

Social media stems from natural interaction and collaboration between individuals to create communities, 
and social media use is grounded in the social learning theory that is rooted in Vygotsky (1978) in which he argued 
that humans develop cognitively and learn through social interaction with other humans using language, cultural 
history, and social context. Dabbagh and Bannan-Ritland (2005) explain that knowledge from the social 
constructivist perspective is distributed and moves from the individual to the community and vice versa to become 
collective as suggested by the situated cognition theory. Flence social media is rooted in both social constructivist 
theory and situated cognition theory (Dabbagh and Reo, 2011). Siemens (2005) proposed a more recent learning 
theory, connectivism, in which he emphasizes the value of what is being learned, and he explains that learning 
happens informally when individuals make sense of information flowing around them unlike the formal learning 
strategies that take place inside the classroom. Fie also emphasizes the fluidity and the growth of learning in a 
cyclical manner as a result of connections between ideas that keep changing across place and time due to networks 
and interactions. Siemens offers a more current learning theory which embraces an information age that offers 
accessible data for all individuals resulting in learner-centered personalized learning otherwise referred to as 
personal learning environments that are supported by Web 2.0 (Dabbagh & Kitsantas, 2012). 

Background Literature 

Despite the fact that undergraduate students prefer to use social media for academic purposes rather than 
personal interaction with their instructors as reported by ECAR (2012), students showed an interest in connecting 
and interacting with other students through social media. This report also suggests that social media and technology 
tools such as open educational resources (e.g. the OpenCourseWare Consortium and the Khan Academy), social 
studying sites (e.g. Cramster, CourseFlero), simulations or game -based learning, e-portfolios, learning management 
systems, online chatting tools, web-citation libraries, and ebooks all contribute positively to the students’ learning 
and academic success. This report provided 10,000 undergraduate students’ perceptions about technology tools that 
help them in academic success. ECAR report does not define what academic success stands for but it referred to 
achieving learning outcomes, preparing for future plans, and getting prepared for the work place. 

Junco, Fleibergert, and Loken (2011) presented the results of a semester long study on students’ use of 
Twitter for academic and co-curricular discussions in a pre -health professional seminar. Students were divided into 
control and experimental groups. The experimental group participated in the following educational activities that 
promote connectivity : continuing class discussions, asking questions in a low-stress environment, engaging in a 
book discussion, receiving class and campus event reminders, receiving academic and personal support, connecting 
with peers and with the instructors, organizing service learning projects and study groups, and completing optional 
and required assignments that require the students to reflect on readings, videos, or sayings and post their tweets and 
then comment on two of their friends’ tweets. The researchers suggested that Twitter had a positive effect on the 
experimental group’s (the group who was using Twitter to communicate with the professor or other students) GPAs 
while both experimental and control groups had the same GPAs in high school. This study also suggested that the 
experimental group showed more engagement with the content than the control group. 

Kelm (2011) described his experience using blogs and media sharing tools with a group of students who are 
part of China Global Connections program that provides students with international business communication 
experience abroad through an MBA program. As part of this experience students are required to post comments (75- 
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100 words) about the daily activities that are scheduled. The students’ comments were to have focused on their 
takeaways about Chinese culture and their awareness of aspects related to the American culture. In this same course 
the instructor asked students to post seven photographs and videos on LESCANTE’s photo-sharing tool and 
YouTube channel, a University of Texas at Austin database that allows students to post media and comment on it in 
a blog format. These social media-based activities encouraged students to engage in discussions, reflections, and 
collaboration to decide on videos to share, and since the program’s purpose was to enrich students’ experience in 
business communication, exposure to the language and the culture and communicating about them is what enriched 
the students’ experiences as suggested by Kelm. 

Blogs can be used interactively as illustrated by Kelm (2011) as well as personal reflection tools. Yang and 
Chang (2012) presented a study in which they describe a learning activity using blogs that requires the experimental 
group to post a blog contribution after each lecture in which they write about course content-related material 
(Information technology). Students in this group were also asked to read and comment on three of their peers’ blog 
contributions. Students in the experimental group who were using blogs in an interactive way were compared to 
another group of students using blogs for personal reflections in an isolated way. As a result, Yang and Chang 
revealed that blogs, when used in an interactive manner can enhance peer interaction, improve critical reflection, and 
increase students’ positive attitude towards academic achievement. 

Cain and Policastri (2011) presented a mixed-methods study on the use of Facebook as a learning activity 
in a pharmacy management and leadership course. They created a Facebook group and invited students optionally to 
join the group. As a result, of the 128 students enrolled in this course (80%) of the students joined the group. The 
purpose of the Facebook group page was to invite guest speakers to submit posts on contemporary management, 
pharmacy, and leadership issues not covered in the course. The students’ participation in the posts was optional with 
no requirements so that the researchers ensure the informality of the nature of the Facebook activity. Data from the 
student survey, exam responses, and student focus groups revealed that students appreciated the informality of the 
activity and the opportunity to connect with professionals in the field and be exposed to “real world” experience. 

Churchill (2009) described a study to investigate the effectiveness of the use of blogs in a postgraduate 
course “in which students were accessing course material, posting reflections, featuring artefacts created through the 
learning tasks, commenting on each other’s contributions and otherwise participating on a regular basis throughout 
the semester” (p. 179). Data was collected through observations of the blog, faculty reflections, student interviews 
and surveys the results of the study suggested a positive learning impact of blogs on student learning. Interesting 
findings from the study proposed students’ perceptions about learning more in that course compared to other courses 
and that blogging facilitated and contributed to their learning. Hence, social media played an important role in this 
course through the interaction that was taking place among students enrolled in one course and the reflection tasks 
that they had to conduct. 

Oskoz and Elola (2011) presented the findings of a study in which they explored the use of a wiki and chat 
tools to complete a writing assignment in a Spanish foreign language face to face course which consisted of 10 
students. The researchers looked at students’ essay, wiki drafts, chat discussions and a questionnaire that was 
administered with the students. Findings suggested that students learned more about foreign language writing 
through the collaborative nature of the wiki and the chatting that took place among the group members. Students 
also reported that the combination of a collaborative site with a chatting tool was very helpful. However, the study 
also suggested that some students did not take full advantage of the affordances that wikis provide and they 
preferred to meet on campus. 

The studies presented above have described and investigated the use of an individual social media learning 
activity in a course; such studies have focused on the use of blogs, wikis, Facebook, or Twitter and have explored 
students’ learning experiences as a result. In general, these studies have demonstrated a positive impact of these 
tools on the classroom environment and on the students’ learning. However, none of the studies analyzed and 
categorized these activities based on the type of learning that students are engaged in while completing these 
activities. Moreover, the studies presented did not examine how social media is used across disciplines rather they 
focused on the use of one social media tool in one course. Research is needed to examine whether social media 
course activities differ across disciplines and Also, several studies presented above revealed students’ perceptions 
or social media use for learning (ECAR, 2012; Churchill, 2009), but it is worth comparing faculty and the students’ 
perceptions of social media use in an educational context to understand how they can best be implemented in these 
settings. Hence this study aimed at addressing the following research question: 

How are faculty’ using social media in higher education, and how do faculty perceptions of the role or value of 
social media use for learning compare with students’ perceptions? 
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The research question was further divided into several questions: 

• In what ways are faculty members using social media in higher education? 

• What are the most common learning activities that faculty assign for students to complete through social 
media? 

• What perceptions do faculty have about their students’ learning using social media tools? 

• What perceptions do students have about their learning through social media tools? 

• How do faculty and students’ perceptions about the use of social media in an educational context 
compare? 


Method 

This study was conducted using mixed methods Greene (2007) describes the originality of mixed methods 
research in creating paradoxes and conflicts which results in new information, and hence the goal of this study was 
to identify the “What” of social media using quantitative methods and the “how and the “why” using qualitative 
methods as Maxwell (2013) suggests. While quantitative methods will provided descriptive statistics, qualitative 
research methods provided the flexibility to explore the topic from different facets due to its narrative nature which 
gave rise to unexpected themes and elaborated on others. 

Research Setting and Participants 

The study was conducted in a higher education institution in Northern Virginia, well known for its 
innovation and diversity . The participants were five faculty members and their respective students, 152 in total, who 
are enrolled in the courses in which social media tools were used. Participants in this study were purposefully 
selected “to provide information that is particularly relevant to the research questions and goals, and that can’t be 
gotten as well from other choices” (Maxwell, 2013, p. 97). They were selected based on their experience in using 
social media, and the different disciplines they teach which included languages, education, and humanities. As for 
the student participants, they were the faculty participants’ corresponding students, a total of 152 students, divided 
among seven academic courses. All of them were undergraduate students except for one group of graduate students 
at the same higher education institution described earlier. 

Data Collection 

Data was collected through faculty interviews and student surveys. Faculty interviews were semi- structured 
with open-ended questions that allowed faculty to talk about their experiences using social media in their respective 
courses. The interview asked eleven questions and one of them had three follow up questions. At the end of the 
interviews, faculty participants were asked if they are willing to follow up with an online survey that would be sent 
to their students to explore their perceptions on the use of social media in general and in their courses in particular. 
The online survey was sent to seven courses because two of the faculty participants were teaching two courses in 
which they use social media; only 21 out of the 152 students responded to the survey despite the several follow-up 
emails and their teachers’ in-class reminders The survey consisted of eight questions, and it explored the students’ 
general use of social media in courses they are enrolled in, and they were also asked to rate how well specific social 
media tools help them learn the course content and how well the learning activities that are implemented through the 
social media tools help them learn. The students were also asked to compare their learning experience in a course 
where social media is not used to a course in which social media used. 

Data Analysis 

Data from interviews were categorized based on pre-existing themes that the interview protocol covered, 
which included course titles, social media tools used, purpose for selecting these tools, description of learning 
activities, perception of impact of social media on students’ learning, professor’s experience teaching with and 
without social media, and challenges. Open-ended question responses to the student survey were also coded based 
on existing themes. Descriptive statistics was obtained from the quantitative sections of the student survey. 

Furthermore, a diagram (Figure 1) was created to analyze the level of learning that the social media-based 
learning activities promote. The diagram sums up three different frameworks that represent different integration 
levels of social media into teaching and learning contexts including Cigognini, Pettenati, and Edirisingha’s (2011) 
2.0 Lifelong learner’s personal knowledge management (PKM) skills as situated in a Web 2.0 era, McLoughlin and 
Lee’s (2011), 2.0 pedagogy framework, and Dabbagh and Kitsantas’s (2012) framework for supporting students’ 
self-regulation in developing their personal learning environments using social media. . All three frameworks share 
common features that suggest different methods of integrating social media in education to promote different 
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learning strategies, but they all emphasize activation of technology skills, facilitating connection with peers and 
instructors, creating or contributing to a practice field, and personalization of learning. 


Figure 1: Diagram for the analysis of social media-based learning activities 
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Results 

The interviews and surveys suggested several findings that address how faculty members are using social 
media across the disciplines and their perceptions about the role of social media in their students’ learning. Some of 
these themes were faculty-only, others were student-only, while others converged. See Figure 2. 

Figure 2: Visual representation of the major themes 
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Faculty Interview Themes 

Types of social media and learning activities. The study revealed that faculty are mainly using blogs, 
wikis, and Twitter as social media tools to support their student learning. While some of these tools have different 
affordances, faculty are using them comparably and differently at the same time. For instance, all three tools are 
being used as reflection tools. Although Twitter is a micro-blogging tool that allows only 140 characters, yet, in 
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some cases it can promote student reflective and metacognitive skills. Blogs and Wikis are both being used for peer 
feedback. However, all these tools are also being used in different ways as Figure 3 explains. 


Figure 3. Findings about the most common used tools and their corresponding learning activities 
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An analysis of 14 social media-based learning activities that the five faculty participants required in their 
courses was conducted based on figure 1 . The analysis revealed that the learning activities are mainly used to 
promote students’ technology skills and to help them connect with people in their fields as well as with faculty and 
their peers. 


Table 1, Example of the Analysis of the Social-media Based Learning Activities 
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Reflection and metacognition. Several learning activities demonstrated a certain level of reflection that is 
facilitated by social media use. Although the use of Twitter or blogs facilitated reflection, however, only in the 
Twitter assignment “Personal Transformation Experiment” (PTE) and in the Wikipedia edits assignment, the 
students thought about their learning and their transformation. The personal transformation Twitter assignment was 
rated by students as helpful for the development of their learning which suggests that assignments which engage 
students in metacognitive activities promote their learning. The professor elaborated on the level of learning that 
students engage in when completing the PTE Twitter activity. 

. . . They design their own experiment, and they say, “Here’s where I am with risk taking, for example, 
now. . .” And they say, here’s what I’m gonna do for the next few weeks to try to sort of become better at 
that in my own way,” and they develop accountability partners to sort of check in with them and they sort 
of, um . . . and at the end they have to say how much they’ve transformed or not. So part of that assignment 
is that they have to tweet almost every day. 

As for the Wikipedia assignment, after the students complete the edits for one of the articles, they have to 
write 750-1000 words to reflect on their learning experience and “discuss the ways in which you [they] might apply 
your [their] new knowledge, of the medium and of the field of digital activism, in the future” as one faculty 
explained in her syllabus. In other tasks that involve blogging such as in “The voice of my story,” students reflect on 
the content of the course and make connections to topics that they find interesting. 

Furthermore, faculty believed that social media allows students to archive and organize their work to create 
a personal repertoire of their work so that they can reflect on their progress; in this respect a faculty participant 
reports that the use of Wikis promotes a sense of achievement in students. This is one of the roles that social media 
plays, and Dabbagh and Kitsantas (2012) illustrate how archiving through social media allows students to reflect on 
their learning and is a step toward guiding students to develop their personal learning environments (PLEs). 
Cigognini et al. (2011) as well discuss meta-reflection and meta-cognition as a strategy to guide students through 
integrating knowledge that helps the e-learner be more successful in the information age. 

Students are already aggregating information from different resources through social media tools which 
allow them to share their perspectives and read about others’ perspectives. One of the students mentioned that social 
media allows him to evaluate his own learning and contributions while other students reported on the quality of 
learning achieved through exposure to different perspectives as discussed in an earlier section of the results. Another 
student described how she could go back and look at her transformation through the Personal Transformation 
assignment on Twitter. All of these examples are evidence of personal learning even if they were not articulated by 
the students in this sense. 

Faculty Interviews and Students’ Survey Common Themes 

Activating technology skills. After analyzing and categorizing the different learning activities that faculty 
assign through social media it was revealed that faculty mainly assign activities that encourage students to use a new 
tool and connect with peers as well as with professionals in the field. Likewise one of the students suggested that 
social media allows her “to step outside of her comfort zone” because she is an older student and the use of social 
media in her coursework forces her to be more “engaged with the technology.” Four out of five faculty members 
reported that some students lacked technology skills and that was a challenge for them to teach students how to use 
the tools before implementing them. Faculty reported that several of their students did not have Twitter accounts 
before enrolling in their courses, or they did not know how to use blogs. One of the faculty for instance stated that 
her students found the Wikipedia editing activity a whole new experience which they were unaware of. . Faculty 
also explained that blogging was considered a new activity to most of the students except for one participant who 
reported that five of her students were already blogging before enrolling in her course. Hence, working with students 
on learning new social media tools and activating their skills is a big challenge for faculty. Faculty believe that it is 
necessary to activate the students’ social media skills because when they graduate they will not be using Blackboard 
in their jobs, rather they will be using social media tools. Similarly, faculty believe that social media gives students a 
sense of what real life looks like and it is “a bridge between the classroom and outside.” Similarly, students see this 
value in social media. One student reported that “classes that utilize social media prepare us for our future careers 
that would use social media tools to communicate in the company.” Another student explained that the course she 
was enrolled in challenged her to learn new technologies which were useful for her on the personal level 

Extending the discussion beyond the classroom. Faculty believe that social networking sites such as 
Facebook and Twitter encourage students to be mindful and think about the topic every day, “seeing the topic 
everywhere they go”. Two of the faculty participants who co-teach a course believed that the Twitter assignment 
“personal transformation experiment” allowed the students to think about the topic all the time because they had to 
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post 14 Tweets about it. One of the students who participated in the “personal transformation experiment” concurs 
with his professors in this aspect when she stated that “microblogging. . . kept my mind on it and kept me 
progressing.” In fact, 100% of the six students who responded to this course survey rated this activity as helpful for 
their learning of the course content which suggests that continuously tweeting about a topic made the students more 
engaged in the topic and aware of it. Moreover, results from faculty interviews and student surveys suggested that 
social media creates a sense of community. Similarly, students believed that the Wiki workspace in particular “has 
created an online writing community of the class” while social media in general “makes a stronger community” that 
extends beyond the classroom. 

The attribute of connecting with professors, peers, or professionals in the field was perceived as useful by 
both faculty and students. All the learning activities analyzed have a “Connection” component. In some activities 
such as “Reflecting on my learning” blog, students connect only with their professor rather than with their peers, but 
this activity allowed the students to reflect on their learning growth. In other activities such as “Language Twin,” 
students have to connect with Spanish native speakers around the world. Similarly, in the Blogging activity “The 
voice of my story” and the Twitter assignment “Connect with informants,” students connect with people from 
around the world. Several students reported the value of building connection with their peers as a method for 
reflecting on their own learning. Students also explained that through online collaboration and interaction with peers 
they got to know them as opposed to a course in which they do not use social media. Hence, connection which is 
central to social media can take several forms in the classroom. Such assignments prepare students for the workplace 
by connecting with people in the field to learn from their expertise and to develop some expectations for their 
careers. 


Facilitating and enhancing students’ learning. Several studies have suggested that social media has a 
positive impact on students’ learning, however it was worth finding out how faculty and students perceive the role 
and value of social media in higher education. As a result, 84.74% of the students who responded to the survey think 
that they learn more in courses that engage them in social media activities. Students were also asked to explain ways 
in which social media has enhanced their learning and faculty were asked the same question about their students’ 
learning. As a result, several factors came to surface as described below. 

To begin with, both faculty and students believed that social media gives students ease of access to material 
where they can refer back to assigned material and read each other’s work or contribute to an online activity on the 
go using a mobile device and using social media applications that they instantly. A student explained that “it is more 
flexible because I don’t need to bring all my books and papers with me, anywhere 1 have internet I can get 
information about the class and assignments.” This ease of access creates a sense of immediacy as suggested by both 
students and faculty, and one of the students explained how easy it was to write about how she was feeling at the 
exact moment through Twitter. 

Collaboration and sharing perspectives that are facilitated by social media play an important role in 
students’ awareness of each other’s work. Several assignments suggest collaboration between students to produce an 
outcome. One faculty participant explained that wikis allow students to share their work and give each other 
feedback whereas before social media existed her students did not use to collaborate on a project in a real sense. 
They used to divide parts of the project among each other and then compile them in one document, and the project 
used to lack coherence. Now with the use of wikis, students collaborated easily and instantaneously and the same 
professor stated that “there you can see a definite improvement, in terms of the goals that I wanted for the 
assignment.” Likewise, students found a great value in sharing perspectives and learning about their peers’ 
perspectives through social media which encouraged peer learning, “It has enhanced my learning of the course 
content through an increased sharing of ideas and perspectives of my own as well as from my fellow colleagues.” 
Faculty explained that the public nature of social media encourages students to produce quality work. The 
students in social media write for an audience rather than for a professor, and they know that many people will read 
their work. Students suggested that using social media has enhanced the quality of their work. Using social media 
makes them reflective of what to write and of the quality of their work. Two of the students elaborate on this idea. 

Posting my work where others can view it has made me more conscious of how I write. Also, the ability to 
see other students' work allows us to learn about each other and from each other, making the class a 
community in which every writer is teaching the others. 

When social media tools are used, however, I realize how my connection with the world enhances m work 
and vice versa. 1 understand now that social media can make our work known or can simply make the 
works and opinions of others known to me while I'm in the process of researching or writing. The use of 
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social media brings forth an interaction between the world and the course that often does not exist 
otherwise. 

Student Survey Themes 

Teacher-student relationship. Students identified a unique student-teacher experience through social 
media activities which was labeled as “equality, as the professor and the students have the same knowledge about 
these tools” as reported by one of the students. In this sense, social media meets the basic foundations of social 
learning theory in which Vygotsky advocates for a reciprocal benefit in student-teacher relationships though not 
equal as that student reported. But as one student explained, social media activities allowed him to get to know his 
professor more, an acquaintance that is hard to make in courses that do not include social media. 

Currency. Many students mentioned how fun and interesting a course that involves social media could be 
as compared to a course that does not. Students believed that courses in which social media is used are more 
relatable to the students’ interest; through these tools, they can interact with peers and look at each other’s work. 
Students believe that these tools also keep them connected to current trends in their field. Therefore, social media 
has a probability of bringing a dynamic to the classroom that does not exist in courses which do not involve social 
media, and as students suggested, it keeps them engaged with the course content. 

Technology and the learning activity. Finally, when exploring students’ learning through social media, 
two factors are involved: the technology and the pedagogy which work hand in hand to produce quality learning as 
suggested by a faculty participant. She believes that making use of social media affordances is what produces 
quality work. The tool by itself does not enhance the learning but rather how the features of a tool are used for a 
specific assignment is what matters. 

I think the design of the assignment needs to take advantage of what social media can do. But if you’re 
using it in exactly the same way than no, there’s probably not any benefit at all. If my students were just 
writing five reflection papers and posting them to a blog instead of what I’m asking them to do which is 
truly a blog, like link around, comment on each other’s stuff - and I can tell them, once the first round of 
blogs is done, if you want your second blog to be in response to somebody else’s first blog, awesome. 
Right? So they love that too. And that’s not something that can happen if you just say five reflection 
papers. Except for post them on a blog instead of typing them. Right?... it’s not just that you’re using social 
. . . It’s not just the platform, it’s the design of the . . . The platform allows you to design the assignment 
differently, and that is what I think enhances student learning. 

Students were also asked to rate the usefulness of social media tools for their effectiveness, and it asked 
them to rate the effectiveness to the social media activities pertaining to their courses. Students were asked to rate 
the tools and the activities on a scale of 1-4 (1 being not useful at all and 4 being extremely useful). The following 
table shows a sample of the ratings. 


300 



Table 3, A Sample of Students ’ Ratings of the Effectiveness of Technology Tools and Learning Activities 


Social media tools 

Average rating of the 
effectiveness of the tool 
by students 

Individual activities 
implemented through 
social media 

Rating of activities by the 
students 

Blogs 

3.30 

Blogging: The voice of my 
story 

Blogging: Students write 
five posts through the 
semester about any topic 
and they have to comment 
on each other’s posts. 

7 responses: 

28.6%: extremely helpful 
28.6%: helpful 
14.3%: somewhat helpful 
28.6%: not helpful 
6 responses: 

33.3% extremely helpful 

50% helpful 

16.7% somewhat helpful 

Wikis 

2.77 

Wikis: Students have 
personalized pages to post 
their assignments; also 
collaborative class 
activities are shared there 

7 responses: 

85.7%: extremely helpful 
14.3%: somewhat helpful 



Collaborative note-taking 
in Wikis 

1 response: 

100%: extremely helpful 



Reading Reflections: 5 
wiki posts 

6 responses: 

50%: extremely helpful 


50%: helpful 


The table above represents only a sample of the learning activities ratings, but the data generally suggests 
that blogs as educational tools were rated as helpful and contributing to the students’ learning as suggested by the 
students. Learning activities implemented through blogs were also considered as useful by students. However, 
Twitter was somewhat considered useful as an educational tool, but ratings of the different learning activities 
showed that these ratings might have depended on whether the students liked the activity or not. Surprisingly, while 
most students considered Wiki activities useful, wikis were rated as somewhat helpful. Hence, it was hard to detect 
through quantitative data students’ experiences whether they rated the tools or the learning activities. 

Discussion 

The study initially investigated two research questions; the first is related to the types of learning activities 
that faculty are assigning through social media and the second question is related to how faculty members and 
students perceive the role of social media in learning. Previous studies have looked at the use of a social media 
activity in one course and compared experimental to control groups. This study added a layer to these individual 
studies by comparing how different social media tools are being used differently or similarly and by looking at the 
learning activities. It also investigated faculty’s perceptions about their students’ learning as opposed to their 
students’ perceptions about social media. It was interesting to look at the different learning activities across 
disciplines and see the levels of interaction that social media tools afford in each of these activities. It was also 
interesting to understand the motives behind faculty’s selection of the different social media tools and the activities, 
and how they perceived implementation in their courses. Although other studies have suggested a positive influence 
on students’ learning as a result of social media, this study shed the light on the “HOW” social media can influence 
students’ learning based on the faculty’s and students’ perceptions. 

The study suggested that the use of social media in higher education plays an important role in preparing 
students for the workplace. Both faculty and students see the value of this aspect. Moreover, this study suggested 
that social media tools are essential in bridging the gap between the classroom and the students’ everyday activities 
or even creating a connection with their peers and people in the field. Students perceived this connection very useful 
in their learning growth since it keeps them mindful of the topics discussed in the class and it gives them different 
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perspectives of the content of the course. This network of information and the connection with peers and people in 
the field give the students an opportunity to learn better in an accessible way through the technology affordances. 
Social media affordances provide students with all the necessary features to collaborate, evaluate, reflect, connect, 
and create learning communities to achieve quality learning. 

Moreover, the results suggested that students perceive a greater learning value in courses that engage them 
in social media activities, however, it was hard to detect through the quantitative data the parody presented in table 
3. While students rated some tools as efficient educational tools, they rated the activities that they completed these 
tools as somewhat helpful and vice versa. Only blogs’ rating concurred with their respective learning activities, but 
it’s evident that activities which engaged students in reflection helped students learn better. On the other hand, the 
open ended questions gave a clearer idea about the value that students perceive in social media. 

Therefore, this study suggests that using social media tools among peers or outside the classroom, 
contributes to the students’ learning. Furthermore, interactive and reflective activities through social media are 
highly valued by students because they feel that they produce quality work for an audience rather than for a 
professor. In addition, despite the perceived challenges of students’ adoption of technology tools as reported by 
faculty members, students highly value learning about tools that they will use in the workplace, and hence faculty 
should carefully select social media that resonate with the students’ future career goals. Finally, faculty should make 
use of all the affordances that a specific social media tool presents while designing a learning activity to promote 
learning. After all social media promotes collaboration, communication, creation, and sharing, and in order for these 
tools to be pedagogically sound, the learning activities designed through social media should take advantage of these 
affordances. 


Limitations 

The study poses several limitations that could be addressed in future studies. The sample size for this study 
was small, and five faculty members using social media tools are not representative of all faculty who use social 
media tools at one institution or in higher education in general. Furthermore, only 21 students responded to the 
surveys, and that is a small size of participants in a survey. More faculty using social media need to be interviewed 
in order to identify a common pattern of learning activities that enhance learning, and more students need to 
complete the online survey. The study investigates the types of instructional strategies that faculty implement 
through social media as well as their perceptions of their students’ learning as a result of using social media. As a 
result, it was hard for most of the participants to compare their students’ learning to students they taught without 
social media. They rather suggested possible benefits and opportunities that social media tools provide learners with. 

Conclusion 

This study compared faculty and students’ perceptions about social media as learning tools, and it 
reinforced findings from previous quantitative studies that social media has a positive impact on student learning 
and engagement. This study has also suggested levels of learning that students engage in through social media-based 
learning activities. More qualitative research is needed to analyze students’ level of learning in using social media 
for educational purpose as well as a comparison of how different social media tools are being used. 
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Introduction 

International students from all over the world come to the United States (Alberts, & Hazen, 2005; 
Sangganjanavanich, Lenz, & Cavazos, 2011) to fulfill their dreams of getting an education and becoming a faculty 
member at an American college or university. Many of those who came to pursue their doctoral studies have 
obtained positions in American institutions of higher education. However, the search for a faculty position has not 
been an easy road for everyone (Ku, Lahman, Yeh & Cheng, 2008). Based on a National Center for Education 
Statistics’ 2008 report, there were 31.222 non-resident faculty in U.S. degree-granting institutions in 2007. This 
number represents 4.4% of the 703,463 faculty (Gahungu, 2011). Although this number does not include academics 
born outside of the U.S. who may have become U.S. citizens through naturalization, or are in the process of 
permanently migrating to the U.S., it suggests there are challenges for international students, who pursue faculty 
positions in U.S. after their doctoral work (Curtin, Stewart & Ostrove, 2012) . One of challenges comes from the job 
search process itself. That is, international students constantly face difficulties and challenges from the preparation 
of their faculty application materials to campus interviews to, ultimately, final negotiations leading to a letter of 
offer. 


The relevance of this issue is especially significant in field of instructional technology because of the 
numerous educational opportunities for international students in the United States. For example, the Association for 
Educational Communications and Technology (2013), the premier “professional association of thousands of 
educators and others whose activities are directed toward improving instmction through technology”, attracts large 
numbers of international students and faculty at its annual conference. Second, a review of past year’s conference 
programs revealed more than 50 percent of authors and presenters had foreign or foreign-sounding names. Further, a 
contingent of members with international lineage is affiliated with the International Division. This division not only 
“facilitates communication among educational technology professionals and students worldwide” but also 
“promotes interactions that transcend international boundaries and cultural lines by mentoring and fostering 
educational endeavors” (AECT, 2013). 

The sharing of stories helps international students understand how things happen and what eventually 
works (Katz, 2010; Moores & Popadiuk, 2011). Hearing stories of job search experiences by international faculty 
members are important to those who follow the similar career paths in the field of instructional technology. The 
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telling of stories is frequently started by someone asking a question. For this article, several questions are asked that 
focus on job searches for faculty positions by international students involving: 

• Perceived significant issues in the job search process 

• High and low points of the job search process 

• Challenges of compiling a job application packet 

• Strategies used to manage a faculty job search 

• Preparation and management of the initial and campus interviews 

• Surprises during the job search process 

• Lessons learned from applying for a faculty position 

Informing international students about the key factors that will result in a successful faculty job search 
needs to be addressed (Cantwell, 2011; Close, Moulard, & Monroe, 2011). This understanding of job search 
challenges will yield a more positive outcome when building a curriculum vita, interviewing for a job, networking 
with fellow professionals, etc. Finally, by discussing experiences surrounding successful and failed job searches 
from the perspective of an international scholarly community, this article provides international students with advice 
about securing a faculty position. This forum will increase international students’ skill level in conducting a 
successful job search and taking actions that will ultimately land them a job. 

Background Information 

This article resulted from two sources of qualitative data generated by an online survey (see Appendix) and 
a panel discussion focused on international students and faculty job search issues. The online survey was completed 
by ten (10) instructional technology faculty members who were former international students in American 
universities. The panel discussion participants involved a smaller and select group (4 international faculty members) 
who completed the online survey. 

The online survey provided demographic information including country of origin, citizenship, gender, age 
range, and length of time as a full-time faculty member. 

Gender and Age Range 

A total of 10 international faculty members (seven females and three males) completed the survey. Also, 
seven reported their ages as 34-to-44 years old while three reported being 45-to-54 years old. 

Country of Origin and Citizenship 

Out of the 10 participants, seventy percent identified the location of their home countries as Asia (Southeast 
Asia = 2, East Asia = 5). Other participants came from the Middle East, Mexico and Africa. Five identified 
themselves as having permanent residency in the United States. One participant reported as a U.S. citizen while four 
as non-U. S. citizens. 

Length of Faculty Service 

As full-time faculty members, the length of service ranges from newly hired to 15 years of service. Five 
participants have worked as full-time faculty from 3 to 10 years. 

Data Collection and Analysis 

The research project collected qualitative data to better understand the international students’ job search 
experiences in the United States. Two sources of data were used to support the writing of this article — responses to 
an online survey and notes from a panel discussion held at an international conference on educational 
communications and technology. Items used in the online survey were generated primarily from the research 
literature and informal conversations with international students and faculty members. 
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After collecting the data, the text-based narratives were converted into electronic formats and coded for 
analysis. Any identifying mark or indicator of who provided a specific response was removed. Finally, the narrative 
data were subjected to a content analysis that identified patterns and trends. 

Findings and Discussion 

The data analysis resulted in the identification of several themes and patterns aligned with the online survey 
questions as well as those addressed during the panel discussion: 1) Perceived significant issues to the job search 
process, 2) High and low points of the job search process, 3) Challenges of compiling a job application packet, 4) 
Strategies used to manage a faculty job search, 5) Preparation and management of initial and campus interviews, and 
6) Surprises during the job search process. Texts in italics are direct quotations of participants’ responses to the 
survey questions. 

Perceived Significant Issues to the Job Search Process 

For the job search process, the participants identified as the most significant issue (see Table 1) - 
interviewing using technology (phone, Skype). On-site (campus) interviewing and writing the letter of application 
were identified as second-and third-most significant issues, respectively. In reviewing the responses, it seems that 
female participants find initial interviewing a bigger issue compared to their male counterparts. 


Table 1. Perceived Significant Issues to the Job Search Process 



Male 

Female 

Frequency 

Initial interviewing (Phone vs. Skype) 

2 

6 

8 

Campus interviewing 

3 

3 

6 

Letter of Application 

2 

3 

5 

Curriculum Vitae 

1 

2 

3 

Questions to ask (Responsibilities, Funding Source, Salary, Resources, 
Search Timeline, etc.) 

1 

1 

2 

Job Ad and Description 

0 

2 

2 

Job Search Presentation 

0 

2 

2 


High and Low Points of the Job Search Process 

When asked, the international faculty participants identified the highs and lows of their job search process 
in an American college or university. Some of the high points reported included: 

1 . Getting an interview. 

2. Getting a job that is as close as possible to a home country. 

3. Contributing to university, college or departmental goals. 

The low points of the job search were many and cut across gender lines as listed below: 

1. Rejection 

• Receiving rejection letters and immediately starting to consider going back to my country. 

• Receiving multiple rejection letters - that was disappointing and terrifying. 

2. Competitiveness 

• Searching and applying for a position that my other (and close) classmates were also applying for. 

• ... sometimes I thought if I'm taking away the chance of[ an] American friend and [then] it's [not] fair to 
them when I apply. 
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3. Language Proficiency 

• Demonstrating a lack in language [English] fluency. 

• Receiving negative comments due to an accent. 

4. Immigration Status 

• Stressing over immigration and legal status “ when I did not get a job ” after graduation. 

• Needing sponsorship from the university’ for my [work] status and I am not sure if school wants to do it. 

5. Qualifications and Credentials 

• Lacking a teaching certificate and/or K-12 teaching experiences. 

• Feeling less competitive because not having enough publication. 

• Unclear of what is expected during the interview process — it was not clear what the committee was 
looking for given my own qualifications. 

Challenges of Compiling a Job Application Packet 

The survey responses identified several issues and challenges specifically on putting together a job 
application packet — from writing cover letters to asking for letters of recommendation. Examples of responses are 
listed below: 

1 . Cover or Application Letter 

• Preparing the cover letter was the most difficult part. I was not sure how much information about my 
internationality I should be giving. Was not sure how receptive the prospective employers would be for a non- 
citizen application. 

• Always running out of time but need someone to proofread before sending out. English writing always 
something challenging so [I] struggled to present myself professionally on paper. 

• I need someone to edit the application, so the minor mistake can be fixed before the material was sent. But, 
it's hard to find an editor. 

2. Letter of Recommendation or Reference List 

• I felt bad asking for recommendation letters or references. When I asked them to faculty members, they 
were nice, and willing to be my references, but it was just hard for me to ask. I think that was from cultural 
differences. 

• I think the biggest issue that I had was to wait for the letters of recommendation. 

3. Curriculum Vitae and/or Portfolio 

• [Un jfamiliarity with the format of a professional resume. 

• It took time to prepare my portfolio, making sure all my work experience is presented. 

4. Other Qualifications 

• Lack of publication record. 

• It was a challenge that I didn 't have K-12 teaching experiences. Many positions that I was interested ... 
required K-12 teaching experiences. I tried to show that I have some experiences working with K-12 
students and teachers, but I don 't think that was enough. 

Strategies Used to Manage a Faculty Job Search 

International students reported multiple strategies in managing their faculty job search to ensure an 
appropriate fit with personal and professional goals. Personal strategies reported by international students in 
managing their faculty job searches were few, such as, 1) checking weather conditions (hot vs. cold) of the 
institution’s location includes location; 2) avoiding the submission of an application to position where they end up 
competing with peers; and 3) being confident, persistent and hopeful given their situation. 
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Survey participants reported multiple professional strategies focused on strengthening their chances of 
getting a faculty position. These strategies are clustered into three areas: Research, Preparation, and Practice. 
Research strategies range from 1) Collecting information about the institution as well as the academic hiring unit 
(Size of the department/ college/university — will I have opportunities to grow given the number of senior faculty; I 
reviewed faculty’ profiles to look for faculty’ with similar research interests. I read about classes and programs.)', and 
2) Identifying alignment between the international student’s personal and professional goals with the vision and 
mission of the hiring institution (Find out if there is a right fit given my own professional goals; Analyze the job 
posting to make sure I'm a good fit.) 

Preparation strategies include 1) Identifying job openings (The only strategy’ I used was to look for jobs in 
the different job databases); 2) Creating good documentation (Consulted with the international faculty’ who got the 
faculty job about writing my resume; I worked really hard on my cover letter to show my experiences in addition to 
my CV), and 3) developing relationships with senior and other professionals (Having a good relationship with my 
professors and external professors; Asked a recommendation letter early). 

Practice strategies involved familiarization of the faculty job expectations from teaching (Preparing my 
portfolio), research (Attended many conferences for the publication record; Submit the articles to the academic 
journals), and service. Also, it is important for international students to practice their interview skills (Practice 
answering interview questions; Ask a lot of help from proofreading my documents, to help rehearse my job talk, to 
discuss potential university; I tried to show that I have experiences in all around areas, research, teaching, design & 
development, etc. They were all listed in my CV, but I felt the bullet points might not fully demonstrate my 
experiences.). 

Preparation and Management of Initial and Campus Interviews 

The application materials were sent and the waiting game started. Most likely an initial interview would be 
initiated if an international student is successful in catching the interest of the search committee and meeting 
minimum qualifications. Initial interviews are usually conducted by phone or by Skype (voice and video 
communication over the Internet). Being successful at this stage means an offer for an on-campus or on-site 
interview. Survey participants report that the interview stage of the job search process is quite stressful. This 
discomfort is not due to the lack of experience or familiarity with the process but being a non-native speaker. 

• Being a non-native speaker, I had concerns about how I would speak/present during both phone and campus 
interview. I just practiced. I did a practice phone interview with one of my friend. I remember her pretending to 
ask how my name would be pronounced. I also did a presentation, which I recorded and watched over and over 
to identify and correct any issues. 

• Of course, I also worried about my English. When I get nervous, my English doesn't come out smoothly. Then 
there was nothing I could do about it except that I kept telling myself not to get nervous. 

• I prepared myself for the interviews/presentation thinking that I was going to do a conference presentation 
(which lam very’ comfortable with). 

Also, what to say and do during the campus interview becomes a critical contributing factor to being 
offered a faculty job. International students double their efforts in preparing and managing this stage of the job 
search experience by being positive, goal-oriented, confident, and well-versed on topics related to teaching, 
research, and service. 

• Trying to be upbeat and positive [during the interview]. 

• Providing detailed responses including specific examples of my work experiences. 

• Reading others ’ resumes and learn[ing\ from the well-written resumes. 

• Getting help [to] rehearse job talks, mock interviews. 

• Being concerned about my visa issue — I just didn 't know how to tell them that I need a H1B visa. I asked 
about it to one of faculty, and he advised me not to say until they ask. So that's what I did. 

• Showing confidence in my research/work. I did deeper research about the faculty’ and the departments to show I 
was familiar with the department's profile and vision. 

• Practicing a presentation with my advisor. 
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• Sending out articles and add[ ing] to the resume as "submitted" to show[the] potential [of the ] publication 
capability. 

• Consulting with both American faculty and international scholars who had job search experiences. 

Surprises During the Job Search Process 

Sometimes the unexpected could serve as a turning point toward success or become problematic to a job 
seeker. International students are not immune to these surprises and have to manage them during the job search 
process. 

• The people were actually much friendly and trying to relate their conversations/experiences to my [foreign] 
identity. One of the faculty members had worked in [my home country] and could speak [my native language] 
during my interview. They liked me and wanted to hire me. 

• Some phone interview questions. 

• The positive reactions from the search committee and the actual results of the job search. The positive feedback 
from the search committee did not land me the job at the end. Higher [administrative] officials had the final 
decision power. 

• They would bring you in even though they already had the final candidate in mind. What a waste of every’one ’s 
time! 

• It was surprising to know that how little people care about I'm an international person. Often they didn 't even 
ask where I was from. 

• Another surprise was how important the good "FIT" to the job is. I applied for jobs that I didn 't feel strongly 
that I would be a good fit, and I haven 't heard from them. There were positions that I felt strongly about the fit, 
and my current job is one of them. 

• I was surprised when I was told, "Your application is great! Your presentation was wonderful! You are very’ 
qualified. But we offered the position to the other candidate. " I wanted to know the reason. 

• Some jobs need the applicant to have the experiences in the US school system and how could this be possible 
for the international students to fit in this requirement? 

• I don 't think this has to do with my residency’ status but with my personality. 

Lessons Learned from Applying for a Faculty Position 

Lessons learned by the participants while applying for a faculty position could be summarized in five areas 
as shown below (see Table 2). Several participants stated that learning about the program, defining yourself as a 
potential candidate, and taking advantage of your international background, are important aspects of looking for a 
faculty position. Taken together, these responses should remind future job seekers how important it is to be well- 
prepared for the job search process. In other words, position yourself as your own best advocate. 
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Table 2. Lessons Learned from Applying for a Faculty Position 


Lessons Learned 

Male 

Female 

1 . Do your 
homework. 

• Know yourself; know what you want; 
find your support group in both 
American and international colleagues 

• Know more about your own culture 
and some interesting aspects of it. You 
may be surprised by how much the 
recruiting faculty’ may already know 
about it and you definitely want to be 
able to talk an informed manner if it's 
brought up. 

• Analyze the job posting to make sure 
it’s a good fit 

• Research the university 

• Learn about the school through the school 
web site before the campus interview 

• Keep factors that are important (such as 
distance to original country, population of 
your ethnicity’, availability’ of your kind of 
food, etc.) in perspective when applying. 

• Play up your unique strength of being 
international (connection and collaborative 
projects with other countries) 

• Find jobs that fit your background (e.g., no 
experience in American education, so find 
departments that do not focus solely on K-12) 

• Understand the school (program) you are 
applying 

2. Practice, 
Practice, 
Practice 

• Practice answering interview 
questions 

• Be modest on the campus interview 

• Practice interviews with a person 

3. Show or say 
something 
good! 

• Prepare a portfolio and presentation 

• Be genuine 

• Clearly explain how you can be a great asset 
to the program 

• Networking, networking, networking! 

• Respect American culture 

4. Have the right 
credentials 


• Have teaching experiences of any kind 

• Have a strong publication record 

• Don 't worry’ much about the fact that you are 
international. As long as you have strong 
research experiences, people don 't pay 
attention to the fact that you are international. 

5. Listen to 
those who 
have the 
experience 


• Seek advice from someone who is already 
familiar with the procedures, be always ready 
and have your application packet updated 
(you never know when you will receive a call) 
and proof-read your documents. 

• Seek help from other colleagues, learned from 
their experiences, and get more job interview 
experience. 


Conclusion and Recommendations 

This paper was prepared to share the insight of faculty members, who have experienced the job search 
process as an international candidate, and to open up a discussion on successful job search strategies appropriate for 
academia. As presented above, the participants identified initial interviewing, campus interviewing, and letters of 
application as significant aspects the job search process. Participants also listed the difficulties associated with the 
low points of the job search process, including receiving rejection letters, worrying about language fluency, 
competing with American students, and having visa/immigration issues. Regarding questions about the challenges of 
preparing a job application packet and for initial and campus interviews, the participants shared openly their 
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experiences and strategies. Many responses to the question about preparing for interviews were related to language 
concerns as non-native speakers. Participants emphasized how important it is to be well-prepared and recommended 
rehearsing, practicing and consulting with advisors. Although it is common for international students to worry about 
their lack language fluency and the challenging visa issues, several participants stated the search committees they 
dealt with did not seem to pay attention to these issues. 

It should be noted that many of the challenges listed in this paper are not exclusive to international students. 
Nearly all doctoral students who pursue a faculty position in academia will face these same difficulties while 
preparing their application and conducting interviews. It is true that international students often have additional 
challenges, but those additions should not be viewed as insurmountable barriers. Overall, our participants’ responses 
reminded us that with thorough preparation it is possible to overcome the challenges and difficulties of applying for 
a job as an international candidate. 
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Appendix 

Online Survey Questions 


1. Country of Origin 

2. Nationality 

3. Citizenship or Residency Status 

4. Gender 

5. Age Range 

6. How long have you been a full-time faculty? 

7. Choose three (3) issues that are significant to your own job search process AS AN INTERNATIONAL PERSON? 

8. What are the high and low points of your job search for a faculty position in an American college or university AS 
AN INTERNATIONAL PERSON? 
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9. What strategies did you use to manage your faculty job search AS AN INTERNATIONAL PERSON? 


10. In retrospect, what strategies would have been helpful in managing your faculty job search experience AS AN 
INTERNATIONAL PERSON? 

11. What issues or challenges did you experience in putting together your job application packet AS AN 
INTERNATIONAL PERSON? 

12. How did you prepare or manage the issues and challenges of the campus interview AS AN INTERNATIONAL 
PERSON? 

13. What were the interview questions that stood out for you (phone or campus)? 

14. What were the “surprises” for you during the job application process AS AN INTERNATIONAL PERSON? 

15. What lessons from can you share with INTERNATIONAL PERSONS who are planning to apply for a faculty 
position in American colleges or universities? 

16. What is your current status? 
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Overview 

International graduate students from all over the world come to the United States to fulfill their dreams of 
getting an education and becoming a faculty member at colleges and universities (Alberts, 2008; Kim, Twombly, & 
Wolf-Wendel, 2012). Many of those who came to pursue their doctoral studies have obtained positions in American 
institutions of higher education (Kim, Wolf-Wendel, & Twombly, 2011; Mamiseishvili & Rosser, 2010). Searches 
for a faculty position, successful or failed, are remembered from stories shared by those who came before a new 
group of hopefuls. International candidates constantly faced difficulties and challenges (Gahungu, 2011; Kuhn, 

1996) from the first day they mail their faculty application materials to campus interviews and the final negotiations 
towards receiving a letter of offer. However, many international faculty members tell future job seekers that the 
search for a position is just the beginning. The real challenge is the conquest of the tenure and promotion process 
especially for international faculty members. 

The understanding of these challenges and the offering of solutions are common to various higher 
education fields. It is obviously a daunting task to manage so it needed a narrowing of scope and led us for analysis 
and sense making to a familiar field of study — instructional technology. The field of instructional technology in the 
United States has drawn numerous educational students and scholars from all around the globe. For instance, the 
Association for Educational Communications and Technology (2013), the premier “professional association of 
thousands of educators and others whose activities are directed toward improving instruction through technology,” 
attracts high number of international students and faculty at its annual conference. A short review of past year’s 
conference programs revealed that there were more than fifty percent listings of foreign or foreign sounding names 
of authors and presenters. Also a contingent of the members with international lineage is affiliated with the 
International Division, which not only “facilitates communication among educational technology professionals and 
students worldwide” but also “promotes interactions that transcend international boundaries and cultural lines by 
mentoring and fostering educational endeavors” (AECT, 2013). 
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Stories of successful tenure and promotion processes experienced by international instructional technology 
faculty members are important to those who follow similar career paths. Stories help understand how things happen 
and what eventually work. The telling and sharing of stories usually start by asking a question. For this article, the 
following questions guided the focus on the experiences of international faculty members seeking tenure and 
applying for promotion: 

• Perceived importance of expectations for those on tenure-track 

• High and low points of the first-year faculty experiences 

• Strategies used to manage the first-year experience 

• Issues or challenge experienced while applying for tenure and promotion 

• Preparation and management of the pre-tenure years 

• Surprises during the tenure and promotion application process? 

• Lessons learned from applying for tenure and promotion? 

The responses to the abovementioned questions identified key factors that provided explanations on the 
outcome of faculty members’ application for tenure and promotion. With better understanding comes good support 
and appropriate resources that address the challenges of demonstrating strong performances as a teacher, researcher, 
grant writer and academic citizen. Finally, in sharing multiple perspectives about successful or failed applications by 
international faculty members, this article provides advice in managing multiple roles given their personal, 
academic, and professional lives. 


Background Information 

This article resulted from two sources of qualitative data generated by an online survey (see Appendix) and 
a panel discussion focused on international faculty, first-year experience on tenure-track, and tenure and promotion 
issues. The online survey was completed by six (6) instructional technology faculty members who identify 
themselves as international faculty in American institutions of higher education. The panel discussion participants 
involved five of the six participants who completed the online survey. The online survey provided demographic 
information including country of origin, citizenship, gender, age range, and length of time as full -time faculty 
member. 

Gender and Age Range 

A total of six international faculty members completed the survey and are equally distributed between 
females and males. All but one participant reported that their ages ranging from 45 to 54 years old. The other 
participants self-reported an age range of 35-44 years old. 

Country of Origin and Citizenship 

Out of the six participants, five identified the location of their home countries as Asia (Southeast = 4, East 
Asia =1). One participant self-reported a home country in the Middle East. Half of the participants identified 
themselves as having permanent residency in the United States and the rest are U.S. citizens. 

Length of Faculty Service 

As full-time faculty members, the length of service ranges from 3-5 years to 16-20 years. Three participants 
have worked as full-time faculty between 5 to 10 years. The remaining three participants reported to have worked 
full-time between 3-5, 11-15 and 16-20 years respectively. 

Data Collection and Analysis 

The research project collected qualitative data to better understand the international faculty experiences 
during their first year on a tenure track and for moving towards tenure and promotion. Two sources of data were 
used to support the writing of this article — responses to an online survey and notes from a panel discussion held at 
an international conference on educational communications and technology. Items used in the online survey were 
generated primarily from the research literature and informal conversations with international faculty members. 
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After collecting the data, the text-based narratives were converted into electronic formats and coded for 
analysis. Any identifying mark or indicator of who provided a specific response was removed. Finally, the narrative 
data were subjected to a content analysis that identified patterns and trends. 

Findings and Discussion 

The findings and discussion sections comprise of six parts: 1) Perceived important expectations for those 
on tenure-track; 2) High and low points of the first-year faculty experience; 3) Strategies use to manage the first- 
year experience; 4) Issues or challenges experienced while applying for tenure and promotion; 5) Preparation and 
management of the pre-tenure years; and 6) Surprises during the tenure and promotion application process. Texts in 
italics are direct quotations of participants’ responses to the survey questions. 

Perceived Important Expectations for Those on Tenure-Track 

When asked about the most challenging expectations for tenure-track international faculty members, most 
of the participants selected research (individual vs. collaboration), scholarship (quantitative vs. qualitative), and 
grant writing (internal vs. external). On the other hand, the participants identified the two top important expectations 
for those on tenure-track (see Table 1) - Research (Individual vs. Collaboration) and Teaching (Discipline vs. Type 
of Institution) respectively. Male and female participants put research as the most important expectations for those 
on tenure-track. 


Table 1. Perceived Important Expectations for Those on Tenure-Track 



Male 

Female 

Total 

Research (Individual vs. Collaboration) 

3 

3 

6 

Teaching (Discipline vs. Type of Institution) 

3 

2 

5 

Scholarship (Quantitative vs. Qualitative) 

1 

1 

2 

Consulting (Service vs. Paid) 

0 

1 

1 

Grant Writing (Internal vs. External) 

1 

0 

1 

Networking (Discipline-specific vs. Interdisciplinary) 

1 

0 

1 


High and Low Points of the First-Year Faculty Experience 

An open ended question asked participants about their high and low points of being the first year faculty in 
American colleges or universities. Some of the high points that the participants reported included: 

1 . Surviving the first year. 

2. Teaching and interacting with students. 

3. Participating in local and cultural activities 

4. Becoming part of the university/college/departmental community. 

5. Having flexible schedule. 

There were many low points reported regarding the first-year experiences on tenure-track, such as, time 
demands for teaching, research and collaboration, teaching challenges (e.g. connecting with students, language 
barriers, and dealing with bureaucracy (e.g., immigration issues and organizational politics). Table 2 lists 
participants’ responses: 
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Table 2. Low Points of the First-Year Experience on Tenure-Track 


“Low Point” Category 


Participant’s Response 


• Organizing teaching materials. 

• Spending tremendous amount of time to prepare for my classes. Being a new 
faculty who is also international, I wanted to make sure to create various 
learning/teaching resources ( e.g ., study guides, detailed descriptions/emails) if 
I'm not fully understood in face-to face class meetings. 

• Spending tremendous amount of time with collaborating with some colleagues 
who also became close friends. Because of close friendship, I had hard time 
saying no to many people to work on projects even though they were not 
productive. I was not able to be direct and I thought this was in part due to 
cultural beliefs. 

Teaching challenges • Building a professional relationship with colleagues/students. 

• Language barriers 

• I cannot quite connect with my students (different from US mainland; special local 
culture that I needed to learn and adapt to); adjusting to department culture (very 
collaborative, more than I was used to) 


Time demands for 
teaching , research and 
collaboration 


Dealing with 
bureaucracy 


• I think the most challenging aspect (low point) in my first-year was the process of 
obtaining my permanent residency (green card) status which is crucial without 
which, tenure is not possible. 

• Swamped and overwhelmed with teaching preparation and organizational politics 
were low points. 


Strategies Use to Manage the First-Year Experience 

International faculty members reported multiple strategies in managing their first-year experience towards 
the path of obtaining tenure. Responses from participants showed similar experiences in finding or having mentors, 
finding balance between teaching and research, and regular communication and asking for clarification, (see Table 

3) 


Table 3. Strategies Use to Manage the First Year Experience 


Personal 


Professional 


Be honest about the challenges and difficulty. 
Ask when in doubt. 

Communicate. Communicate. Communicate. 

Too many times, I assume things are the way 
they are supposed to be without questioning, 
when questioning would have helped. 

While teaching is always a priority because 
your students/courses needed attention, I would 
definitely recommend striving hard to keep a 
balance and devote time to research/publication 
as well. 


Finding a mentor who could advise me regarding 
which projects to be involved in and what projects to 
avoid would have been helpful in terms of time 
management. 

Had a solid mentor who would provide solid, just in 
time advice and support. Set up regular meeting with 
mentor and stick to it. 

Learn[ed] from the colleagues/Seek[mg] help from the 
colleagues/mentors. 

Dedicate some writing time; less idealistic in my 
teaching; avoid getting involved in organizational 
politics. 
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Focusing on publications and presentations. 


Good teaching skills and caring attitude toward 
students. 

Preparing teaching to the best I can. 
Communicate^ d] with the administrators. 


Issues or Challenges Experienced while Applying for Tenure and Promotion 

Participants identified several issues or challenges that impacted their behaviors and dispositions toward the 
tenure and promotion process: 

• Long application process required the patience. 

• Find people to proofread my applications, to help me rehearse my job talk, to answer my questions about the 
process. 

• Understanding historical issues and current issues of American education systems, esp. in the Teacher Education 
program 

• I was fortunate that I obtained my green card was before I went up for tenure but have heard of horror stories of 
international colleagues who had tremendous challenges with their green card application which then held up 
their tenure process. 

• Finding external reviewers was challenging. I was not sure how to and whom to approach faculty’ to ask if they 
consider being an external reviewer for my application. 

Preparation and Management of the Pre-Tenure Years 

The pre-tenure years happen so fast given all the adjustments that new international faculty members have 
to make at both personal and professional levels at their institutions. The first-year on tenure-track needs to focus on 
understanding both the curriculum and culture of the department, college and university. At the same time, 
international faculty members have to determine the expectations for teaching, research and service commitments. 

Second year usually kicks off the preparation for a pre-tenure review. Survey participants provided sound 
advice on getting organized, such as, 

• Creating a box, binders or folders to collect or organized all materials and artifacts. 

• Viewing past portfolio samples 

• Analyzing teaching evaluations statistically and putting them in a grid. 

• Analyzing teaching evaluations qualitatively and identify thematic patterns aligned with good characteristics of 
the best professors. 

• Reflecting on teaching practices based on peer review, teaching evaluations, and students' feedback and 
identifying the changes made as supported by data. 

• Writing narratives bit by bit for the dossier given expectations on teaching, research/scholarship, and service. 

Also, participants suggested the need to connect with American and international faculty members as well 
as campus resources: 

• Learning and talking to people as much as possible about ideas on one ’s teaching, scholarship, and service 
involvement. 

• Seeking support from faculty development centers by attending workshops 

• Consulting with former advisor regarding how to prepare the application package and finding the external 
evaluators. 

• Seeking people who could be mentors as well as forming support group with other international faculty in similar 
situations. 
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• Seeking peers for support who may speak the same language. 

• Have good friends to cheer you on issues and challenges encountered. 

Surprises During the Tenure and Promotion Application Process 

Sometimes the unexpected could serve as a turning point towards success or become problematic to those 
seeking tenure and/or applying for promotion. International faculty members are not immune to these surprises and 
have to manage them during the pre -tenure years. Some of the surprises identified by participants included 

• I guess it is way more time consuming than one expects. 

• Nothing except for the permanent residency status issue. 

• It's good to have invested in good deeds in the early years of my pre-tenured years. 

• Institutional politics 


Lessons Learned from Applying for Tenure and Promotion 

Several lessons were shared by participants related to the application for tenure and promotion: 1) support 

from an individual or group; 2) “blowing your own horn"; 3) More writing and less presenting; 4) Balancing 

between productivity and collegiality; and 5) understanding the tenure and promotion system of the institution. 

Comments related to these lessons are listed below: 

• Find a few buddies to work on this together. Read each other’s document, encourage one another. This is going to 
take time. Get your colleagues from your department to be on your side. 

• I think the biggest challenge as an international faculty in writing a tenure/promotion dossier is getting used to 
"blowing your own trumpet". Coming from an Asian upbringing where we are always taught to be humble, I 
found it challenging to brag about my accomplishments but this is precisely what one needs to do in a 
tenure/promotion application. 

• Write, write, [and] write. Submit, submit, and submit. Looking at what I have presented at conferences over the 
years, I realized I could have done less [participation in] conferences and more publications. 

• Productivity, collegiality and always improving that CV. 

• Understanding the system of getting tenure and promotion in an institution and selecting the university with open 
and fair system 


Conclusion and Recommendations 

This panel focused on the tenure and promotion processes experienced by six international faculty 
members in the field of instructional technology. The faculty members completed an online survey regarding issues 
they encountered and strategies to cope with the issues. The reporting of international faculty members’ experiences 
provided practical suggestions to those who follow similar career paths. Future research examining international 
faculty members’ experiences at different stages (e.g., early, mid, senior) and using higher number of participants 
will contribute to the understanding that emerged in this study. 
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Appendix 

Online Survey Questions 

1 . Country of Origin 

2. Nationality 

3. Citizenship or Residency Status 

4. Gender 

5. Age Range 

6. How long have you been a full-time faculty? 

7. Choose three (3) issues that are significant to you during your first year as AN INTERNATIONAL FACULTY 
MEMBER? 

8. What are the three most IMPORTANT expectations for a tenure-track INTERNATIONAL FACULTY 
MEMBER? 

9. What are the three most CHALLENGING expectations for a tenure-track INTERNATIONAL FACULTY 
MEMBER? 

10. What are the high and low points of your first-year experience as an INTERNATIONAL FACULTY MEMBER 
in an American college or university? 

11. What strategies did you use to manage your first-year faculty experience AS AN INTERNATIONAL PERSON? 

12. In retrospect, what strategies would have been helpful in managing your first-year faculty experience AS AN 
INTERNATIONAL PERSON? 

13. What issues or challenges did you experience in your application process for tenure AS AN INTERNATIONAL 
PERSON? 

1 4. How did you prepare or manage the issues and challenges during your pre-tenure years AS AN 
INTERNATIONAL PERSON? 

15. In retrospect, what would you have done differently in preparing for tenure AS AN INTERNATIONAL 
PERSON? 

16. How similar and/or different is the process of earning promotion compared to receiving tenure AS AN 
INTERNATIONAL PERSON? From Assistant to Associate? or Associate to Full Professor? 

17. AS AN INTERNATIONAL PERSON, what were the “surprises” for you during the application process for 
tenure and/or promotion? 

18. AS AN INTERNATIONAL FACULTY MEMBER, what lessons can you share with those who are planning to 
apply for tenure and/or promotion in an American university? 

19. What is your current status? 
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Abstract 

This empirical study investigated five students’ online inquiry-based learning experiences. A 
generic qualitative approach was employed. The study found that students’ inquiry-based learning was 
a multifaceted process that was complex and challenging due to a range of interrelated activities and 
different strategies for performing those activities. Students needed scaffolding almost every step of 
their inquiry learning process. The study suggested a series of specific scaffolds that could be provided 
to support successful online inquiry-based learning. 

Online Learning and Inquiry-Based Learning 

Recent 10 years have witnessed the explosive growth of Web-based online courses in colleges due to the 
expansion of the Internet and advancement of other interactive communication technologies (USnews, 2013). 
However, the quality of online learning is a major concern in this new landscape (Sims, Dobbs, & Hand, 2002). 
Associated with the low quality is students’ negative perception of online learning experiences include, but are not 
limited to, a feeling of isolation, limited interactions with instructors and peers, and time management problems, etc. 
(Middleton, 1997). 

To address these problems, many researchers advocate applying constructivism to distance education 
(Jonassen, 1994; Jonassen, Davidson, Collins, Campbell, & Haag, 1995; Moller, 1998). Inquiry-based learning, as 
one of the constructivist pedagogies is a promising pedagogy that can improve online learning. Generally speak, 
inquiry-based learning begins with posing a problem or question, followed by generating and pursuing strategies for 
investigating, collaborating, reflecting on, and justifying the solutions of the problem or answers to the question, and 
communicating the conclusions (Sandoval, 2005). Research indicates that engaging in inquiry can 1) improve 
students’ learning of their disciplines (Krajcik, Blumenfeld, Marx, & Soloway, 2000), 2) reduce the discrepancy in 
learning between genders and races (White & Frederiksen, 1998), and 3) nurture critical thinking and information- 
processing skills (Blumberg, 2000). 

However, online students’ inquiry-based learning experiences have seldom been investigated. Weerasinghe 
and others’ empirical study showed that encouraging discussion could result in meaningful and deep learning 
(Weerasinghe, Ramberg, & Hewagamage, 2012). Change, Sung, & Lee (2003) conducted a study of 17 college 
learners’ experiences for collaborative inquiiy -based learning. Their study reveals that adult learners practicing 
inquiry-based learning have problems similar to those of middle and high schoolers, such as lack of motivation, 
knowledge and reflection (Edelson, Gordin, & Pea, 1999). Their findings also confirmed the importance of authentic 
tasks, collaborative learning, and social interactions. However, their research study has weaknesses. The whole time 
used for students’ inquiry-based learning is only 80 minutes, while meaningful inquiry-based learning often requires 
sustained periods and systematic approaches (Scardamalia & Bereiter, 1994). Thus, the soundness of their findings 
based on such a short period needs further confirmation. 

Based on the above analysis of inquiry-based learning in online contexts, we can conclude that students’ 
online inquiry-based learning process has almost not been explored. Investigation of adult learners’ online inquiry- 
based learning experiences is needed to add to the literature base. My study was designed to provide rich and 
detailed description of students’ semester long online inquiry-based learning experiences. More specifically, I will 
investigate how students use different strategies to conduct their online inquiry-based learning projects. The research 
questions are: 1) How do students choose their inquiry topics? 2) How do students generate their questions? 3) How 
do students explore and collect information related to their inquiry topics? 4) How do students respond to their 
collected information? 5) How do students present their inquiry findings? 
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Qualitative Methodology 


The research site was a graduate online course in instructional technology in the Southeast in a large public 
university in the spring of 2006. The goal of this course was to develop students’ information literacy. This online 
class blended mainly asynchronous work in the WebCT environment with synchronous meetings twice a week in 
the chat room. Online instructions and interactions was through e-mail, WebCT discussion board, virtual chat room, 
telephone, etc. Mandatory face-to-face sessions were scheduled at the beginning of the semester. 

Five participants were selected on a volunteer basis. Data generation was from in-depth interviews and 
students’ documents. Each student participant was interviewed five times. Documents included participants’ 
journals and final papers, learning logs and other documents they created during the process of their learning (see 
Table 1 for participants’ profiles and Table 2 for participants’ project overview). 


Table 1 Participants’ Profiles 


Name 

Current Profession 

Age 

Demographics 

Degree 

Bette 

High school language arts 
teacher 

40-45 

Caucasian Female 

B. S. in English 

Mary 

Elemental school media 
specialist 

45-50 

Caucasian Female 

M. S. in Psychology 

Grace 

Middle and high school 
Spanish teacher 

50-55 

Caucasian Female 

B. S. in Spanish 

Charles 

Elementary teacher 

45-50 

Caucasian Male 

M. S. in Early Childhood 
Education 

Emma 

Physician 

40-45 

Caucasian Female 

M. D. in Medicine 

Hannah 

Middle School English 
Teacher 

50-55 

Caucasian Female 

M. S. in Library Media 


Table 2. Project Overview 


Name 

Topic 

Essential Questions 

Secondary Questions 

Final 

Presentation 

Format 

Bette 

Mind/body 

connections 

How can the mind be 
used to affect change in 
the body and 
specifically, how can 
hypnosis be used to 
affect in the body? 

N/A 

PowerPoint 

Presentation 

Mary 

Early 

Christian 

Church 

history 

How has the addition of 
man-made organization 
taken us away from the 
model demonstrated by 
the community of the 
first believers (church)? 

(a) How is the church to be governed, 
according to Scripture? (b) How did the 
early church organize? (c) How do these 
compare? 

Poster 

Grace 

Plan a 
London trip 
with her 
niece 

How can 1 help my niece 
develop a deep 
understanding of 
London’s history and 
culture? 

(a) Which two books will be most useful 
to us while traveling? (b) Which sights 
and activities will interest Hannah and 
provide some insight into London history 
and culture? (c) How can we prepare so 
that Hannah develops a greater 
understanding of London history and 
culture? (d) How should we record our 
trip and new understandings? 

Birthday gift 
cards 
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Charles 

Blue Ridge 

Parkway 

camping 

Why am I attracted to 
the Virginia section of 
the Blue Ridge Parkway 
as a vacation spot? 

None 

Itinerary map 

Emma 

Family 
houseboat 
trip on the 
Suwannee 
River 

Why is the Suwannee 
River a good place for 
our family and friends to 
take a houseboat trip? 

(a) Which activities will help us all enjoy 
the trip? (b) How will we decide which 
supplies to bring? (c) Why did Stephen 
Foster write about the Suwannee River? 

Handbook 

Hannah 

Hot tub 

How do benefits 
outweigh the 
disadvantages of owning 
a hot tub? 

None 

Chart 


Inductive analysis was employed in this qualitative study to investigate the research questions because 
inductive analysis is appropriate for discovery of important patterns and themes, as well as for providing detailed 
description from the data (Patton, 2002). 


Findings 


Ql: How do Students Choose Their I-Search Topics? 

Data analysis revealed that participants employed multiple methods and combinations of these methods to 
determine an I-Search topic, including: (a) long-term interest, (b) usefulness or practicality, (c) foundation, (d) 
balance, (e) stress reduction, (f) doable, and (g) other people’s suggestions. The findings indicated that participants’ 
prior knowledge might or might not influence their topic choice, but they did influence participants’ perceptions of 
their topics, be it a difficult topic or easy one. It was also noticeable that narrowing down a topic was an important 
and challenging step for most participants. For example, Grace did not realize this was a necessary step. Mary did 
not narrow down her topic enough to produce a good focus. This study illustrated that online teacher need to help 
students to narrow down their topics. 

Q2: How do Students Generate Their I-Search Questions? 

Qualitative data analysis revealed that the six participants generally used a two-step method to generate 
their essential or primary research questions. The first step was to determine a focus within the selected topic. The 
second step was either to generate the essential questions around the focus or to generate essential questions to cover 
several other questions. 

Participants’ generation of their sub-questions proved to be a very complex process. Three participants did 
not clearly generate their sub-questions at the beginning, but let the research progress make their sub-questions 
gradually emerge, while one participant was very specific about what her sub-questions should be. Participants also 
utilized a wide range of methods to generate their sub-questions, including: (a) personal interests, (b) questions 
scaffolding, (c) other already generated questions, (d) What I don ’t Know column questions from their pre- 
notetaking sheet, (e) consolidation of several questions into one question, and (f) impressions gained from scanning 
materials. 

Problems that participants experienced in this step included: (a) creation of factual questions instead of 
higher-order questions; (b) research questions that did not bring participants much new knowledge; (for example, 
One participant, Charles, asked a higher-order question that began with why: Why am I attracted to the Virginia 
section of the Blue Ridge Parkway as a vacation spot? But he thought that he already had the answer to this 
question. That meant his research question would not bring him any change in that answer. Whatever new 
knowledge about the VA section of the Parkway that he would find he would not use to answer his essential 
question.) (c) incompatibility between essential questions and sub-questions, and (d) questions that were based on 
false assumptions and needed revising, for example, Mary’s case illustrates this situation. Her questions were 
designed to find a church organization model from early Christian church history and the Bible that today’s church 
could follow. But after exploring related information, she realized that such a model did not exist in early church 
history. 
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In short, the overall findings about students’ question generation in this study imply that creating questions 
is a not just a single act, but consists of a number of intellectual operations and involves various factors that might 
influence the question generation. Only scaffolding students to create higher-order questions is not enough at this 
step. The other issues needing consideration include students’ prior knowledge, new knowledge that could be gained 
through posing a particular type of question, different natures of the two types of questions, and compatibility 
between essential questions and their sub-questions. 

Q3: How do Students Explore and Collect Information Related to Their I-Search Topics? 

The findings illustrated that participants also employed diverse strategies to explore relevant resources for 
their I-Search projects. The three most common venues for resources were the Internet, library and other person’s 
lived experiences gained through interviews. Their information was also in different formats, including journal 
articles, web pages, maps, books, tapes, digital video disks, and video. They also used different methods to organize 
their information, including bookmarking Internet pages, printing out and/or sorting resources, and using word 
documents to save short notes and links for information. 

Data also illustrated that participants used different criteria to choose a resource for their topics. Reliability 
was the number one criterion, although reliability had different meanings for different participants. Another criterion 
was the participant’s personal preferences, such as personal camping philosophy and personal belief. 

Qualitative data showed that exploring information was not just one step in the process, but iterative 
throughout most stages of the I-Search. In this process, some participants used generic to specific strategies to search 
for relevant information. Without an effectively narrowed topic, participants might also be tempted to collect too 
much information thinking that all of it would help them answer their research questions. The qualitative analysis 
also revealed that some participants needed scaffolding to use their research questions to guide their information 
searching. 

In short, findings about students’ exploring relevant information delineate that it is a complex learning 
phenomenon involving students’ prior knowledge, personal preferences, different stages of inquiry-based learning, 
and the research question and its modification. Those are the issues we need to consider when scaffolding students’ 
during this step of inquiry-based learning. 

Q4: How do Students Respond to Their Collected Information? 

Participants in this study made meaning out of their collected information through making a reflected 
response to their collected information. These were recorded in their double-entry drafts and in their journals. Most 
of the responses could be classified as knowledge construction responses. The first type of response was when 
participants realized that a particular piece of information could inform their research questions. The other responses 
included: (a) confirming knowledge, (b) activating prior knowledge, (c) changing knowledge, (d) enriching 
knowledge, (e) identifying a knowledge gap, and (f) validating knowledge. Although participants did not have 
difficulty making responses to their collected relevant information, one participant seldom made knowledge changes 
or validated her information findings. 

The overall findings in this step indicate that students make a wide range of different types of responses, 
when they confront their collected information. Knowledge change response is only one of them. This may imply 
that scaffolding should be more diverse to support various types of responses besides knowledge changes. 

Q5: How do Students Present Their I-Search Findings? 

There were two types of factors that influence participants’ choice of format for their final product. 
Individual issues included the level of participants’ technology skills. Contextual issues included available time and 
influence from other people. This implies that support with some technology skills instruction should be available to 
students to facilitate selecting a format that best fits their projects. In addition, enough time should be allocated to 
students to help them present their findings. 
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Practical Implications 


Consider Students’ Prior Knowledge 

This study finds that students’ prior knowledge influences students’ topic selection and perceptions of their 
topic. The finding suggests that when students make a decision about their topic, the instructor needs to encourage 
students to think about how much knowledge they have about their topics, and compared to their prior knowledge, 
how much knowledge they need to know in order to complete their inquiry tasks. The instructor should not 
encourage students to take a task, where they will not bring much new knowledge once they finish their projects. 
Narrowing Down within a Topic Scaffolding 

This study reveals that, although selecting a topic is not a problem for participants, further narrowing down 
within a topic to a clearly defined specific focus associated with higher-order thinking is a challenge for most 
participants. As my findings showed, this step not only influences essential question generation, but also influences 
later information exploration. Some participants know the need to further narrow down their topic, but do not know 
how to do that. For example, Bette wanted to narrow her broad topic of mind/body connections, but she did not 
know how to do that until she read an article about hypnosis and determined to focus on it. In cases such as this one, 
the instructor could facilitate students’ ability to think about different approaches that could be taken for their topics 
and to select one clearly defined focus from many possible approaches. 

Scaffolding Question Generation 

The findings of this study show that participants use different approaches to generate their essential 
questions and sub-questions. The literature tells us that essential questions and sub-questions serve two different 
goals in the inquiry-based learning process (Flakkarainen & Sintonen, 2002; Tallman & Joyce, 2006). The primary 
question, or essential question, sets the goal for inquiry, and should be in the format of a higher-order question 
(Hakkarainen & Sintonen, 2002). The sub-questions should be designed to help answer the essential questions 
(Tallman & Joyce, 2006). Both qualitative findings and the literature indicate these types of questions should be 
scaffolded differently. For example, emphasize higher-order questions when students generate their essential 
question, but emphasize how sub-questions could be used for helping answer the essential question. The findings 
from this study also show that participants could generate essential questions and sub-questions which are not 
compatible to each other. For example, Bette’s essential question and sub-questions pointed in different directions. It 
is essential to keep compatibility between the two types of questions (Tallman & Joyce, 2006). Therefore, the 
instructor needs to help students generate related primary and secondary questions. When students change any of 
them, the instructor should require students to revise their other type of questions appropriately, if applicable. 

This study also illustrates that some questions might change after participants have adequate knowledge 
about their topics. Even if they gain enough background knowledge to define their question in more specific terms 
through background reading, it still happens that questions sometimes need changing. Participants’ original 
questions might be insufficient or too briefly answerable. When exposed to more information, they want to revise 
their research questions. For example, Grace reformulated some of her sub-questions at the end of the I-Search 
process based on what she discovered. This is also confirmed by other study findings that students may change their 
research questions as they explore more relevant information (Persky, 1992). The instructor might encourage 
students to rethink their research questions, especially their primary question, as they master more relevant 
knowledge. 

Information Exploring throughout the Entire Process 

The findings of my study also tell us that most participants explore relevant information throughout the 
entire process. Based on the findings from this study, I suggest that the instructor should emphasize the entire 
process of exploring relevant information. The instructor should also scaffold students in setting different goals at 
different stages of inquiry. In addition, the instructor can suggest different search strategies for different students, 
according to their prior knowledge about their topics and the stage of their inquiry. This instructional intervention is 
in accordance with what other researchers (Quintana, Zhang, & Krajcik, 2005) suggested, which is to help set search 
steps for students, as well as help them reflect on their previous searches, “in order to determine what their 
subsequent searches should entail" (p237) in their inquiry-based learning. Finally, the instructor should foster 
reflection on one’s personal beliefs and preferences and their influence on information exploration. 
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Information Exploring for Answering Research Questions 


One different feature of exploring information for inquiry-based study is that after questions are generated, 
the exploration should be constantly focused on answering those questions, instead of finding everything related to 
the students’ topic. Qualitative analysis from this study shows that some participants, like Grace, were not able to 
refer consciously to their research questions, when they searched for relevant information. Therefore, as Tallman 
and Joyce suggested (2006), the instructor needs to guide students to connect their information with their research 
questions to make their search more purposeful and meaningful, instead of collecting any interesting information. 
Encouraging Multiple Responses 

Participants made meaning out of their collected information. My study also shows that some participants 
seldom made knowledge change responses. This type of response could foster knowledge reconstruction, which is 
emphasized by most current inquiry-based learning models. For example, Zuckerman, Chudinova, and Khavkin 
(1998) proposed that students should constantly check and revise their model about a natural phenomenon, when 
they find new data that cannot support their theories. Encouraging students to make this type of response can help 
students make thoughtful efforts to change their own knowledge and be more reflective during this process. 

Summary 

Inquiry-based learning is important because it fosters higher order thinking and meaningful knowledge 
construction. However, current literature lacks substantive knowledge about students’ online inquiry-based learning 
experiences. The weakness in the literature will hinder further relevant research and practice. This qualitative study 
aimed to fill the gap in the literature by investigating students’ online learning experiences. It is my hope that the 
findings of this study can shed light on the relevant literature and provide a guideline for further study into inquiry- 
based instruction. 
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Abstract 

This research measured the creative skills of 1000 Minnesota schoolchildren, compared it to performance on 
standardized achievement tests and examined the relationship between measured creativity and intelligence in our 
current school systems. Measured creativity has declined in school children since 1991 and this decline will have a 
greater impact on economic vitality than shortfalls in math or science. Methods, results, and analysis of the research 
are presented. Research was funded through a program to support regional competiveness. 

Introduction 

Over fifty years ago, at the University of Minnesota, Dr. Paul Torrance created what remains the "gold 
standard" for measuring creativity. As part of his initial research. Dr. Torrance tested thousands of school children in 
the Minneapolis area and rated their creativity. Subsequent long-term research on the test results found that 
creativity was the most salient factor in lifelong success. 

Creativity is essential to the economic health of a region; innovative capabilities of a community or nation 
are what spark economic engines. Nations, countries, and states around the world have all begun to seek to expand 
their creative capability; Singapore and Taiwan have creativity as part of their national curricula. There are national 
efforts in Denmark and England; Oklahoma City has declared itself to be a World Creativity City. 

Today, the defining skills of the previous era— the "left brain" capabilities that powered the 

Information Age— are necessary but no longer sufficient. And the capabilities we once disdained or 

thought frivolous— the "right-brained" qualities of inventiveness, empathy, joyfulness, and 

meaning— increasingly will determine who flourishes and who flounders (Pink, 2004, p. 22). 

At the same time, the value of creativity and innovation has been strongly recognized in business. The most 
recent IBM worldwide survey of chief executives lists creativity as the most important skill for a contentious and 
dynamic future, far outpacing any other management skills. 

Early in the history of Minnesota, we became a state with a strong knowledge economy. We know that the 
current economic health of the metropolitan area developed from a number of factors; support of a quality 
educational system, our social engagement, and numerous innovations and inventions created in the State. We know 
the names and the inventions as part of a legacy granted to Minnesota, and we recognize the value of maintaining 
creativity and innovation as building our future. For example, 3M, a leading Minnesota company, needs a new 
innovative product every week to remain viable. 

The jobs of the knowledge economy will not be based solely on what you know, but on the new ideas, on 
synthesis, and on connections that can be generated. 

At the same time, our state and national educational system has focused more and more on a narrow set of 
characteristics; standardized tests that deal with the fundamentals. The United States performs relatively poorly on 
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international tests in math and sciences, and browbeats itself; Minnesota evaluates all state schools on a series of 
achievement tests that focus on reading, science and mathematics. It is implied that these are the attributes that will 
lead to a successful society. 

Some counties are seeking less reliance on standardized testing. For example, East Asian nations are 
consciously moving away from strict standardized testing to encourage more creative learning just at the time the 
United States is placing more emphasis in the reverse direction. As a Chinese educator said: "You’re racing toward 
our old model. But we’re racing toward your model, as fast as we can." (reported in Bronson, 2010) 

One consistent argument about standardized testing is the validity of the measure; whether test performance 
in writing and math equates to long term success for the individual or society. In contrast, creativity testing has been 
correlated to life long innovative productivity in areas such as patents, businesses started, and internationally 
recognized achievement. Longitudinal research by Plucker (1996) on the early creativity test subjects finds the 
correlation to lifetime achievement three times stronger for creativity than for general intelligence. 

We addressed three primary questions through this research. First, we sought to understand how creative 
our contemporary schoolchildren are. According to research by Kim et. al. (2011), creativity scores of school 
children have been declining since 1991. 

The second question is an examination of how the current Minnesota Comprehensive Assessment test 
corresponds to creativity measurements. In general, intelligence tests do not directly correspond with creativity tests. 
As creativity is a significantly higher indicator of future achievement, this comparison was an important evaluation 
regarding the validity of more comprehensive academic tests. 

The third question the research addressed was the concept of the achievement gap. Currently, in Minnesota 
and nationwide, there are significant differences between minorities and the dominant population in terms of 
achievement as measured by standardized testing. How this corresponds to creativity will be examined; do those 
students who do not perform well on the MCAs excel on creativity metrics? 

Research Venue 

Research was conducted in a single, large suburban school district in Minnesota and within multiple middle 
and high schools across the district. The district currently enrolls about 39,000 students from 14 different 
communities and is the largest in Minnesota. While urban and rural districts were considered for the research 
location and would have different characteristics, the scale and demographics of the district were representative of 
the entire state. Within the district, demographics and socio-economic status varied significantly by school location 
while curriculum remained relatively consistent. Six district middle schools were used for testing, and five high 
schools were tested. 

The creativity research was conducted in concert with the research department of the district. Achievement 
scores were paired with creativity scores. Prior to release to the research team, all personally identifying features 
will be removed by the school district. As we were working directly with the research department, a greater range of 
possible areas for investigation were available. Comparisons were made between creativity scores and demographic 
data such as race, economic status as evidenced by free and reduced lunch, and noted disciplinary citations. 

The district where testing occurred had approximately 39,000 students; 78% were white, 10% black, 4% 
Hispanic, 6% Asian, and 1% Native American. 32% of the students in the district receive free or reduced price 
lunch. 7% have limited English proficiency. 13% receive special education. 

Methodology 

In order to moderate administrative challenges and to conduct the testing with a minimum of disruption to 
ongoing educational programs, tests were administered to intact, randomly selected home rooms. Tests were 
administered to 47 eighth grade classrooms and 39 eleventh grade classrooms. Each test booklet was labeled with an 
administrative code for individual students. Teachers from each school were trained in the administration of the tests 
and they administered the tests in their regular classrooms. 

Creativity Testing 

The Torrance Tests of Creative Thinking, Figural Version A was used to measure creativity. The Torrance 
Tests were developed and first published in the early 1960's and are widely used to test creativity. The test uses a 
series of three forms of visual prompts to spur drawn responses from test subjects. The test takes a total of thirty 
minutes to administer in three equal segments of ten minutes. In the district schools, tests were administered over 
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three days due to scheduling limitations; this is an accepted method for test administration according to the test 
protocol, which was verified with the test publisher. 

The Torrance Tests were scored by Scientific Testing Services, the publisher of the test. This provided a 
high level of reliability for the test evaluation and comparison with a large existing database of test scores organized 
by age, grade, and region. Test results were provided to the research team as well as the school district. Individual 
test results were provided to schools for possible distribution to students. 

Scoring for the Figural Test is evaluated on a wide variety of metrics that examine the ability to produce 
numerous and novel ideas as well as skills in abstraction, elaboration, and synthesis. Basic and brief descriptions of 
the lead metrics are: 

• Fluency [FL]: The ability to develop multiple ideas to a given stimulus 

• Originality [OR] : Generation of unusual or unique ideas; based on statistical infrequency 

• Abstractness of titles [TI] : relates to synthesizing and organizing aspects of thinking; capturing the 

essence of the information presented. 

• Elaboration [EL]: imagination and development of ideas. 

• Resistance to closure [CL]: moving past the simple, direct answer to more original ideas. 

Under this testing system, each metric receives a raw score which is derived from thethe number of 
answers or elements that can be identified under the evaluation terms. Raw scores are then converted to a 
standardized score with a median of 1 00 and a standard deviation of 20 using a database of previous tests and 
evaluated by grade and age. 

Minnesota Comprehensive Achievement Tests 

The regular Minnesota Comprehensive Achievement tests were administered on their regular annual 
schedule. These tests are taken by all eight and eleventh graders in Minnesota. This test battery includes 
measurements of math, science, and reading. They are computer scored and students are rated as meets, exceeds, 
partially meets and does not meet standards. Scores are also provided as to student percentile rank. Data from the 
MCA tests was provided to the researchers with all identification removed. 

Additional data were provided by the district office including school, art courses completed, gender, 
discipline citations, and status of limited English proficiency and eligibility for free and reduced lunch. 

Aggregation of Data 

Each individual was assigned a randomized identification number and Torrance, MCA, and other data were 
aggregated for each individual by the district research office. While the comparison between the standardized 
achievement scores and the Torrance scored were the primary interest of the research, other information was 
included to provide a richer understanding of the nature of creativity and achievement. 

Analysis 

General levels of creativity were examined by calculating the means of both raw and standardized scores 
for all students for each of the creativity metrics. Raw scores were also compared between grades 8 and 1 1 . Product 
moment correlations were calculated for eight variables; the five metrics from the Torrance Tests and the three 
MCA scores. A Principal Component Analysis was completed for the eight cognitive measures as well. Descriptive 
statistics were used to compare the six middle schools and to examine the differences between genders. T-tests were 
conducted for comparison of groups formed on the basis of disciplinary action, completion of art classes, and 
eligibility for free and reduced lunch. Results are presented below accompanied by brief commentary on the 
findings. 

Results 

Results from the Torrance Tests and subtests were analyzed in comparison to national samples based on 
age and grade. 8th graders scored about 1/2 standard deviation above the population mean in terms of Fluency, the 
ability to generate multiple, divergent answers, and Originality, generating unusual or unique answers. They also 
scored 1/2 SD lower in terms of Titling, synthesis or abstraction of ideas, and Elaboration imagination and 
development of ideas. 

On a percentile basis, scores for Fluency and Originality were above the mean at 68 th and 60 th percentiles; 
scores for Elaboration and Titling were below the mean at 36 th and 34 th percentiles; Resistance to Closure was at the 
66 th percentile, and the average scores were at the 55 th percentile. 
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TTCT Figural-A Standardized scores, 8 th grade, grade-based scores 


Table 1 


Fluency SS 

Originality SS 

Elaboration SS 

Titling SS 

Closure SS 

Average SS 

113.74 

106.44 

89.84 

87.84 

109.22 

101.41 


"SS" is used for Standardized Score. 


Descriptive Statistics for Creativity Subtests for the Sample of 8th Grade Students 

Table 2: Descriptive Statistics of Creativity Thinking Ability’ for 8 ,h Grade Students (N=996) 




Min 

Max 

Mean 

Std. 

Deviation 

Grade- 

Based 

National 

Percentile 

Age-Based 

National 

Percentile 

Subscale 

Scores 

Fluency 

0 

40 

24.02 

9.41 

79 

78 

Originality 

0 

39 

16.66 

7.03 

71 

71 

Elaboration 

3 

18 

7.75 

2.72 

38 

52 

Abstractness of Titles 

0 

22 

6.18 

4.02 

29 

38 

Resistance to Premature 
Closure 

0 

21 

13.41 

4.02 

62 

62 

Total Score 


3 

116 

68.03 

20.83 

- 

- 


Table 2 shows the general descriptive statistics for the creativity subtests. According to the raw score 
statistics, 8 th grade students performed best (in terms of National Percentile) on the Fluency subtest (M= 24.02, SD 
= 9.41) and worst on the Abstractness of Titles subtest (Mean=6.18, SD=4.02). 

Compared with the Grade-Based National Percentile, 8 th grade students performed above the national 
average on subtests of Fluency, Originality, and Resistance to Premature Closure, and below the national average 
on subtests of Elaboration and Abstractness of Titles. 

Using Age-Based National Percentiles, 8 th grade students performed above the national average on subtests 
of Fluency, Originality’, and Resistance to Premature Closure, and below the national average on subtest of 
Abstractness of Titles. Student performance on Elaboration is almost equivalent to the age-based national average. 

In addition, scores of eighth graders were compared to scores of 1 1 th graders. In comparison with Table 1, 
Table 3 shows 1 1 th graders are less creative than the 8th graders, both in terms of raw score and in terms of 
standardized grade based scores. They are strong and weak in the same areas. Restated, in general, eleventh graders 
scored lower than eighth graders in virtually all areas. 

All ethnic groups followed the performance of the district as a whole on creativity tests, with highest scores 
on Fluency, and lower scores on synthesis and Elaboration. Numbers of ethnic minorities were fairly small and 
probably limit the statistical analysis at this level. However, significant differences were found between genders. 


TTCT Figural-A Standardized scores, 11 th grade 

Table 3 


Fluency SS 

Originality SS 

Elaboration SS 

Titling SS 

Closure SS 

Average SS 

104.93 

102.00 

92.77 

93.45 

106.11 

99.87 


"SS" is used for Standardized Score. 
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Descriptive Statistics for Creativity Subtests for the Sample of 11th Grade Students 

Table 4: Descriptive Statistics of Creativity’ Thinking Ability for 11 th Grade Students (N=762) 




Min 

Max 

Mean 

Std. 

Deviation 

Grade- 

Based 

National 

Percentile 

Age-Based 

National 

Percentile 

Subscale 

Scores 

Fluency 

0 

40 

20.72 

9.17 

58 

57 

Originality 

0 

39 

14.78 

6.93 

54 

53 

Elaboration 

3 

18 

7.16 

2.47 

40 

39 

Abstractness of Titles 

0 

22 

6.32 

3.96 

42 

41 

Resistance to Premature 
Closure 

0 

21 

12.88 

4.34 

61 

61 

Total Score 


3 

116 

61.86 

26.87 

- 

- 


Descriptive Statistics for Creativity Subtest Scores for Male and Female Students and a 
Comparison of the Subtest Means for the Male and the Female Participants 

Table 9 Gender Comparison for 8 th Grade Students 




Mean 

Std. Deviation 

National Percentile 

F test 

P 

Grade- 

Based 

Age- 

Based 

Fluency 

Male 

23.78 

9.82 

79 

78 

0.70 

.40 

Female 

24.28 

8.99 

79 

78 

Originality 

Male 

16.03 

7.20 

65 

66 

8.24 

.00 

Female 

17.31 

6.80 

71 

71 

Elaboration 

Male 

6.91 

2.42 

23 

36 

105.59 

.00 

Female 

8.60 

2.75 

53 

65 

Abstractness of Titles 

Male 

5.62 

3.61 

29 

38 

20.61 

.00 

Female 

6.76 

4.32 

40 

47 

Resistance to Premature 
Closure 

Male 

13.01 

3.97 

62 

62 

10.04 

.00 

Female 

13.82 

4.03 

70 

69 


According to Table 9, the 8 th grade male and female students perform equivalently on the Fluency subtest, 
and they are both above the national average. For the Originality subtest, the 8 th grade male and female students 
perform above the national average. Female students perform significantly higher than male students ( F= 8.24, p < 
• 01 ). 

For the Elaboration subtest, the 8 th grade female students perform above the national average; whereas, the 
male students perform below the national average. Both male and female students have scores below the national 
average on the Abstractness of Titles subtest. Specifically, the 8 th grade female students have an advantage over the 
male students on this subtest ( F = 20.610 p < .01). The 8 th grade male and female students are above the national 
average on the Resistance to Premature Closure subtest with the female students having a significant advantage (F 
= 10.04, p < .01) on this subtest. 

Generally speaking, the 8 th grade girls tend to have a distinct advantage on each creativity subtest. 

District performance on the Minnesota Comprehensive Achievement tests 

At the eighth grade level, 57% of the students were proficient in Math, 54% were proficient in Reading, 
64% were proficient in Science. Significant score changes occurred in Reading due to changes in the test used. State 
averages for proficiency were: Math 59%, Reading 54%, Science 44%. 

Product-Moment Correlations among Eight Cognitive Variables 

Scores on the creativity subtests and the Minnesota Comprehensive Achievement tests were then examined 
for correlation to examine relationships between the two testing protocols. At the time of this writing, individual 
MCA scores were only available for eighth grade students. 
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We have found that the creativity scores varied in their correlation with the Minnesota Comprehensive 
Achievement Tests. Correlations between the creativity test scores and the three MCA scores were examined and 
three observations were made. First, as would be expected from the literature, the first two metrics of creativity. 
Fluency and Originality were highly correlated at .79. Torrance himself reports an correlation of .81 and this is 
consistent with recent meta-analyses. 

Second, the Minnesota Comprehensive Achievement scores were strongly correlated among themselves at 
.77, .72, and .69. This is addressed further in the Principal Components discussion that follows. 


Table 5: Correlation Matrix for the Creativity’ Subtest and Scholastic Achievement Scores 



Fluency 

Originality 

Elaboration 

Titling 

Resistance 
to Closure 

Science 

Math 

Reading 

Fluency 

1 








Originality 

79 ** 

1 







Elaboration 

32** 

44 ** 

1 






Titling 

14** 

.28** 

48** 

1 





Resistance 
to Closure 

4g** 

.45** 

.36** 

.36** 

1 




Science 

Q9** 

.16** 

.31** 

.30** 

yj ** 

1 



Math 

Q9** 

.15** 

.30** 

29 ** 

.13** 

77 ** 

1 


Reading 

07** 

.15** 

.35** 

.30** 

14** 

72** 

.69** 

1 


Note. * p_< .05, ** p < .01. 


Third, while Fluency, Originality, and Closure were not strongly correlated with achievement scores, the 
other two measures, Elaboration and Abstractness of Titles, were moderately correlated. This may indicate a 
connection between these higher order thinking skills (elaboration, synthesis, and abstraction) and achievement 
scores. These two scores were also moderately correlated with the other three creativity scores as well. 

Discussion 

Some findings stand out from our analysis of the collected data. First is the irregular performance of 
students on the Torrance Creativity Test. The means for students in both the eighth and eleventh grade were 
significantly higher than the national mean for Fluency and Originality. This indicates students in general were able 
to generate more ideas and the ideas they developed were viewed as more original than national levels for their age 
and grade. Flowever, mean scores for two other metrics, Elaboration and Abstractness of Titles, were significantly 
lower, indicating less skill in the higher order thinking skills of elaboration, development of ideas, synthesis of 
information, and abstract thought. Combined, this may indicate a focus on quick solutions or answers to challenges, 
and a lack of skill in developing more complex or complete ideas. It could be hypothesized that this is a result of 
substantial standardized testing and preparation for testing with less emphasis on the development of higher order 
thinking. 


"Everyone agrees that creativity is a key skill for the twenty-first century, but we're not teaching 
our kids this skill. We've become so obsessed with rote learning, with making sure that kids 
memorize the year of some old battle. But in this day and age that's the least valuable kind of 
learning. That's the stuff you can look up on your phone!' [Kyle Wedberg, CEO, New Orleans 
Center for Creative Arts as quoted in Imagine, by Jonah Lehrer.] 

Perhaps the most substantial finding was the moderate correlation between the two higher order creativity 
metrics, Elaboration and Titling with both creativity and the standardized test results. While the three MCA scores 
were well correlated and the two initiation-type creativity scores were highly correlated, these other scores provide 
the connective tissue between creativity and achievement. We would hypothesize that additional educational 
development in the areas of elaboration, synthesis, and abstraction would indirectly support advances in 
achievement scores. This is consistent with findings from Vaughn (2005), where arts oriented courses were 
correlated with increased SAT scores. 
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